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Executive  Summary 


In  February  2003,  the  Naval  Research  Advisory  Committee  was  charged  by  Mr.  John 
J.  Young,  Jr.,  Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition)  to 
conduct  a  study  on  technology  acquisition  reform.  The  major  difference  between  this  study 
and  others  on  acquisition  reform  is  that  this  study  focuses  on  ways  to  more  smoothly  inject 
new  technology  into  acquisition  programs. 

The  Department  of  the  Navy  wants — and  needs — to  exploit  technological  advances 
more  rapidly.  The  ability  to  field  technologically  sophisticated  systems  and  use  them  effec¬ 
tively  is  a  distinctive  asymmetrical  advantage  the  United  States  enjoys  over  its  adversaries, 
and  anything  that  increases  our  rate  of  innovation — whether  the  innovations  originate  in  gov¬ 
ernment,  industry,  or  abroad — works  to  the  advantage  of  our  Sailors  and  Marines. 

But  this  challenge  can  be  frustratingly  difficult  to  meet.  Persistent  problems  stand  in 
the  way.  Cultural,  bureaucratic,  and  programmatic  obstacles  bedevil  the  acquisition  of  tech¬ 
nology.  The  culture  that  surrounds  the  acquisition  system  makes  it  difficult  to  bring  new 
technology  into  that  system.  This  system  remains  very  conservative:  structured  to  avoid 
fraud,  conflict  of  interest,  and  risk,  it  discourages  the  research  and  development  community 
from  taking  risks  and  seizing  opportunities.  Designed  to  eliminate  risk  and  meet  cost  and 
schedule  constraints,  the  system  rightly  treats  unproven  technology  as  a  risk  to  its  programs. 
Whether  the  acquisition  consumer  of  technology  is  a  Naval  Program  Manager  or  a  contrac¬ 
tor,  that  consumer  is  disinclined  to  accept  risky  new  technology  (however  high  the  payoff  it 
promises)  for  fear  of  failure  (and  the  results  of  failure). 

Implicit  in  Sea  Power  21  is  an  imperative  to  speed  development  of  new  concepts  and 
technologies  and  to  streamline  the  procedures  that  facilitate  their  rapid  implementation.  The 
six  recommendations  of  this  study,  listed  below,  will  shorten  the  time  required  to  respond  to 
the  warfighter’s  needs. 

Institutionalization  of  a  Rapid  Technology  Acquisition  Team  concept  would  establish 
an  ability  to  respond  rapidly  to  the  warfighter’s  unique  and  immediate  needs,  bypassing  the 
more  deliberate  acquisition  process.  The  approach  would  insure  that  logistical  support  is 
provided  and  maintained  for  items  delivered  through  this  accelerated  process. 

Similarly,  a  Direct  Reporting  Program  Office  for  Disruptive  Technology  would  incu¬ 
bate  promising  technologies  until  they  are  ready  to  be  handed  over  to  an  established  program. 
This  office  would  also  provide  a  home  for  disruptive  technologies  emerging  from  discovery 
and  invention.  Since  these  potentially  transformational  technologies  are  not  linked  to  exist¬ 
ing  acquisition  programs,  they  need  an  alternative  path  to  maturity. 

Requiring  a  science  and  technology  project,  whether  or  not  it  directly  supports  a  Fu¬ 
ture  Naval  Capability,  to  identify  an  acquisition  advocate  when  it  reaches  Technology  Readi¬ 
ness  Level  (TRL)  3,  and  to  have  a  signed  Technology  Transition  Agreement  at  TRL  4,  will 
facilitate  a  more  rapid  maturation  and  acceptance  of  technology.  It  would  focus  resources  on 
science  and  technology  that  have  a  customer,  and  it  would  give  the  customer  responsibility 
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for  program  oversight  beyond  TRL  4.  The  net  result  should  be  a  higher  yield  from  the  sci¬ 
ence  and  technology  program. 

Using  established  metrics  and  TRLs  to  measure  a  technology’s  readiness  for  insertion 
into  acquisition  programs  would  improve  the  chances  of  program  success.  Not  only  will  spe¬ 
cific  programs  benefit,  but  the  consumer’s  confidence  in  science  and  technology  will  also 
increase,  to  the  general  improvement  of  the  research  and  development  culture.  Contractual 
and  personnel  incentives  to  temper  risk  aversion  and  encourage  technology  insertion  should 
further  improve  the  maturation  and  infusion  of  technology.  The  synergy  of  these  initiatives 
will  encourage  proper  assumption  of  well-managed  risk  to  meet  the  needs  of  the  Fleet  and 
Force. 


The  test  and  evaluation  process  is  often  described  as  a  principal  obstacle  to  rapid 
handover  of  technology  from  acquisition  community  to  the  warfighter.  In  order  to  be  most 
efficient  and  effective,  the  test  and  evaluation  process  must  complement  evolutionary  acqui¬ 
sition  and  spiral  development.  An  additional  study  is  recommended  to  determine  ways  to 
better  accomplish  test  and  evaluation. 


A  technology  knowledge  base  is  essential  and  should  be  established.  The  acquisition 
customers’  program  is  best  served  if  a  repository  of  technology  development  efforts  were 
available  to  improve  identification  of  candidates  for  technology  insertion,  which  in  turn  may 
accelerate  delivery  of  mature  technology  to  our  Sailors  and  Marines. 
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Terms  of  Reference 


This  NRAC  study  will  examine  current  approaches  to  managing 
DON  acquisition  programs  with  a  particular  emphasis  on  technology 
acquisition.  This  study  will  also  examine  alternative  approaches  tested 
by  other  departments,  agencies,  and  countries.  Specifically,  this  NRAC 
study  will: 


•  Review  examples  of  new,  emerging,  and  experimental  technology 
acquisition.  For  example,  NMC1  lessons  learned,  CTTO  lessons 
learned,  In-Q-Tel,  Army  venture  fund,  DoD  and  Navy  venture  fund 
plans,  UK  R&D  privatization. 

•  Investigate  acquisition  alternatives  studied  by  USD(AT&L),  and  others. 

•  Recommend  procedures  and  opportunities  to  streamline  and 
improve  technology  acquisition  subject  to  regulatory  restrictions. 


The  panel  emphasized  identification  of  actionable  and  unique  recommendations  to 
enhance  technology  acquisition  for: 

■  Timely  anticipation  of,  or  response  to,  both  traditional  and  asymmetric  threats, 

■  Responsiveness  to  the  needs  of  the  acquisition  community — the  primary  con¬ 
sumer  of  science  and  technology’s  products,  and 

■  Effective  delivery  of  war  lighting  capability  to  maintain  our  technological  ad¬ 
vantage. 

The  full  text  of  the  Terms  of  Reference  is  in  Appendix  A. 
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Panel  Participants 


CHAIR 
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Rear  Admiral  Lewis  A.  Felton  USN  (Ret) 

Private  Consultant 

Dr.  Stuart  E.  Johnson,  Ph.D. 

Professor,  Technology  &  National  Security 
National  Defense  University 

Vice  Admiral  E.  R.  Kohn,  USN  (Ret) 

Private  Consultant 
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Professor  Emeritus 


|  Years  of  Experience  ) 

Operational 

Program 

Management 

Technology 

Development 

Acquisition 

Workforce 

Industry 

Government 

84 

105 

82 

87 

71 

252 

The  Panel’s  members  brought  a  rich  and  robust  experience  based  in  several  critical 
areas  that  impact  technology  acquisition.  This  panel  contained  deep  operational  expertise, 
diverse  program  management  experience,  significant  technology  development  knowledge, 
and  major  acquisition  acumen.  It  is  also  noteworthy  that  these  experiences  stem  from  both 
industry  and  government  perspectives. 
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Fact 

Finding: 


Briefings  and  Visits 


United  Kingdom  Trip: 

•  MoD  -  Public  Private  Partnership 

•  DPA  -  S&T  Acquisition  Strategy 

•  DSTL  -  Overview  of  changes  in  research  and 
acquisition 

•  QinetiQ  -  Transition  from  DERA  to  QinetiQ 

•  Rolls  Royce  -  Corporate  Technology  Acquisition 

Other  Discussions  &  Briefings  Reviewed: 

•  Commercial  Technology  Transition  Officer 

•  Executive  Director,  SPAWAR  Systems  Center 

•  FNC  Brief  to  DASN(RDT&E) 

•  CNR  Corporate  Brief 

•  GAO  Report  on  Technology  Management  Best 

Practices 

•  OSD  Brief  on  Transitioning  S&T  Programs 

•  OSD  Management  Guide  to  Technology  Transition 

•  Air  Force  Brief  on  Applied  Technology  Council 

•  Army  Brief  on  Technology  Transition  Workshop 
Assessments  &  Recommendations 


y 


The  Panel  conducted  a  series  of  fact-finding  meetings  to  gain  Naval,  Defense,  Indus¬ 
try,  and  Congressional  perspectives. 

The  Panel  also  visited  the  United  Kingdom,  where  it  conducted  particularly  useful  in¬ 
terviews  with  senior  representatives  of  the  United  Kingdom’s  defense  research  and  develop¬ 
ment  community.  These  British  officials  had  particularly  valuable  insights  into  the  privatiza¬ 
tion  of  research  and  development. 

Finally,  the  Panel  conducted  other  discussions  and  gathered  other  material  as  needed. 
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Fact  National  Defense  University  Research 

Finding: 


•  NDU  research  library  conducted  a  broad  search 
of  recent  studies  (1997  to  present)  on  technology 
acquisition 

-  DoD:  DSB,  other  DoD 

-  FFRDC:  CNA,  RAND,  IDA 

-  Congress:  GAO,  CRS,  CBO 

•  Conclusions  and  recommendations  were 
reviewed  by  the  panel 


The  National  Defense  University’s  research  librarians  conducted  a  broad  search  of 
recent  (since  1997)  studies  of  technology  acquisition.  They  found  relevant  studies  from  a 
wide  spectrum  of  the  defense  community: 

•  Department  of  Defense:  Defense  Science  Board,  Office  of  the  Secre¬ 
tary  of  Defense  Inspector  General,  all  three  Service  Departments; 

•  Federally  Funded  Research  and  Development  Centers  (FFRDCs): 
Center  for  Naval  Analyses,  RAND  Corporation,  Institute  for  Defense  Analyses,  Car¬ 
negie  Mellon  University  Software  Engineering  Institute; 

•  Congress:  General  Accounting  Office,  Congressional  Research  Ser¬ 
vice,  and  Congressional  Budget  Office. 

The  research  librarians  summarized  the  studies’  conclusions  and  recommendations 
for  the  panel.  The  Panel  reviewed  the  summaries  to  establish  a  baseline  of  knowledge  de¬ 
rived  from  the  work  of  other  expert  teams,  thus  avoiding  duplication  of  effort  and  gaining 
insight  from  other  studies.  Several  of  the  sources  the  National  Defense  University  located 
bear  directly  on  Naval  technology  acquisition. 
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Technology  Acquisition  Context 


Ops  Sea  Power  21  4^  ■C=| 
c=^T>  Capability  Packages  4^  C1 
Acquisition  Programs 


Acq 


A 


=>  ►  Future  Naval  Capabilities  4^  <= 

Enabling  Capabilities 


S&T 


A  >  S&T  Exploitation  &  Delivery  4^ 
S&T  Discovery  &  Invention 


Requirements 
■4  Technology 


c=4>  Joint  &  Coalition  Requirements 
COTS  Technology 


A  continuum  runs  from  desired  operational  capabilities  to  the  basic  science  that  will 
provide  warfighters  with  enhanced  future  capabilities. 

Technology  ought  to  serve  some  useful  end.  This  should  be  obvious,  but  the  fre¬ 
quency  with  which  one  sees  technologies  pursued  with  no  apparent  operational  end  in  view 
indicates  that  it  is  not.  The  Naval  process  for  technology  acquisition  recognizes  this  in  its 
emphasis  on  requirements  flowing  down  from  the  operators  to  the  Program  Managers  (PM), 
engineers,  and  scientists,  to  be  met  with  capabilities  flowing  back  up  through  the  acquisition 
system.  Conceptually  clear  and  well  structured,  this  system  is  in  fact  riven  by  seams.  We 
see  these  open  between  the  operators  and  the  resource  sponsors,  between  the  resource  spon¬ 
sors  and  the  acquisition  community,  and  between  the  acquisition  community  and  the  science 
and  technology  (S&T)  community. 

To  achieve  mission  success  all  constituent  communities  must  work  together  seam¬ 
lessly.  As  requirements  flow  from  the  operational  community  to  ultimately  drive  S&T  in¬ 
vestments,  we  need  technology  to  flow  into  operations  in  order  to  maintain  our  warfighters’ 
technological  advantage. 

So  far  this  is  a  purely  Naval  view,  but  Naval  operations  are  normally  conducted 
jointly  with  other  services  or  in  coalition  with  allies.  These  bring  additional  requirements 
that  further  complicate  matters.  At  the  same  time  rapidly  accelerating  advances  in  commer¬ 
cial  technology  need  to  be  integrated  to  give  our  warfighters  further  technological  advan¬ 
tages. 


The  Panel  believes  its  recommendations,  if  adopted  and  acted  upon,  will  strengthen 
communications  among  these  communities.  They  will  also  further  build  confidence  that  the 
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S&T  community  can  deliver  technology  at  the  speed  and  maturity  required  by  the  acquisition 
community  to  field  winning  technological  advantages  for  our  warfighters. 
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Challenge: 


More  rapidly  exploit  government  and 
commercial  technology  advances 


Confounding  Issues: 

-  Cultural  environment 

-  Bureaucratic  barriers 

-  Programmatic  issues 

-  Inconsistent  incentives 

-  Accelerating  technology 
development  cycle 

-  Technology  transition  from 
S&T  to  Acquisition: 
“Valley  of  Death” 


Goat:  Maintain 
Technological  Advantage 


The  Department  of  the  Navy  (DON)  wants — and  needs — to  exploit  technological  ad¬ 
vances  more  rapidly.  The  ability  to  field  technologically  sophisticated  systems  and  use  them 
effectively  is  a  distinctive  asymmetrical  advantage  the  United  States  enjoys  over  its  adversar¬ 
ies,  and  anything  that  increases  our  rate  of  innovation — whether  the  innovations  originate  in 
government,  industry,  or  abroad — works  to  the  advantage  of  our  Sailors  and  Marines. 

But  this  challenge  can  be  frustratingly  difficult  to  meet.  Persistent  problems  stand  in 
the  way.  Cultural,  bureaucratic,  and  programmatic  obstacles  bedevil  the  acquisition  of  tech¬ 
nology.  The  culture  that  surrounds  the  acquisition  system  makes  it  difficult  to  bring  new 
technology  into  that  system.  This  system  remains  very  conservative:  structured  to  avoid 
fraud,  conflict  of  interest,  and  risk,  it  discourages  the  research  and  development  community 
from  taking  risks  and  seizing  opportunities.  Designed  to  eliminate  risk  and  meet  cost  and 
schedule  constraints,  the  system  rightly  treats  unproven  technology  as  a  risk  to  its  programs. 
Whether  the  acquisition  consumer  of  technology  is  a  Naval  PM  or  a  contractor,  that  con¬ 
sumer  is  disinclined  to  accept  risky  new  technology  (however  high  the  payoff  it  promises)  for 
fear  of  failure  (and  the  results  of  failure). 

The  S&T  part  of  the  research  and  development  community  has  shown  the  obverse  of 
this  problem:  a  tendency  to  work  on  technology  for  its  own  sake,  without  proper  attention  to 
risk  mitigation  or  realistic  operational  needs.  Despite  efforts  toward  cultural  change  across 
the  research  and  development  community,  both  syndromes  persist. 

Bureaucratic  obstacles  and  programmatic  barriers  can  retard  the  pace  of  advance.  In¬ 
centives  are  available — and  more  have  become  available  with  recent  reforms — to  move  tech¬ 
nology  forward  into  acquisition,  but  such  incentives  are  applied  inconsistently  and  often  fail 
to  achieve  their  intended  result.  Presently,  few  programs  have  built-in  contract  incentives  for 
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the  contractor  to  insert  new  technology  in  a  reasonable  cycle,  and  even  fewer  programs  offer 
the  acquisition  work  force  incentives  to  pull  new  technology  into  a  program.  This  amounts 
to  a  large  barrier  in  the  path  of  technology  acquisition. 

Our  acquisition  system  is  so  vertical,  and  relies  so  heavily  on  consensus-building  that 
one  negative  vote  can  kill  a  good  initiative.  Comptrollers  in  particular  often  interfere  with 
budgets  that  allow  for  technology  insertion  by  sweeping  up  cost  savings  from  new  technolo¬ 
gies  to  fund  shortfalls  in  other  programs.  The  rigidity  of  the  Planning,  Programming,  and 
Budgeting  System  (PPBS)  also  inhibits  the  introduction  of  new  technology  that  appears  off- 
cycle  from  the  Program  Objective  Memoranda  (POM)  and  Future  Years  Defense  Program 
(FYDP)  planning  cycles. 

Technology  itself  is  advancing  at  an  increasing  rate,  and  we  have  not  been  as  success¬ 
ful  as  we  might  in  bringing  the  latest  advances  into  fielded  systems.  Technology  matures  at 
various  rates  and  in  different  cycles.  Moore’s  Law,  for  example,  continues  to  govern  the 
computer  industry,  which  now  cycles  at  a  rate  of  about  eighteen  months.1  By  way  of  con¬ 
trast,  propulsion  technology  tends  to  undergo  major  changes  every  fifteen  to  twenty-five 
years.  With  the  increased  global  availability  of  new  technology,  and  the  push  of  United 
States  industries  to  move  more  rapidly  to  market,  several  of  the  longer  cycles  of  technology 
change  are  getting  shorter.  Since  much  of  our  advantage  lies  in  our  ability  to  work  new  tech¬ 
nology  into  systems,  to  stay  ahead  of  our  adversaries  we  must  infuse  new  technology  more 
rapidly. 


Finally,  Naval  research  and  development  still  suffer  from  a  “Valley  of  Death”  in 
which  potentially  promising  new  technologies  languish  and  die.  This  gap  between  S&T  on 
the  one  hand  and  acquisition  on  the  other  has  narrowed,  especially  in  the  Future  Naval  Capa¬ 
bilities  (FNC)  process,  but  it  nevertheless  persists  to  some  degree.  The  panel  believes  that 
the  Valley  of  Death  is  a  self-inflicted  wound  that  could  be  closed  through  more  closely  cou¬ 
pled  transition  mechanisms  informed  by  the  discipline  of  Technology  Readiness  Levels 
(TRLs). 


1  Moore’s  Law  states  that  the  number  of  transistors  that  can  be  placed  in  a  production  integrated  circuit  doubles 
roughly  every  eighteen  months. 
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Diversity  of  Technology  Categories  and 
Findings.  Cycles  Complicate  Technology  Acquisition 

•  Globally  available  technology  s7 

•  “Credit  Card”  acquisition  / 
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Consider  this  graphic  representation  of  technology  development  cycles.  All  these 
technology  categories  presently  advance  at  an  increasing  rate — none  is  mature  or  stable.  Be¬ 
cause  new  capabilities  are  increasingly  available  globally,  many  of  our  adversaries  have 
moved  to  a  credit-card  acquisition  model — they  simply  buy  whatever  they  find  they  can 
make  use  of.  Our  national  advantage  in  using  technology  comes  from  our  ability  to  integrate 
new  technologies  into  systems  rapidly.  (Having  said  this,  there  is  also  no  reason  why  we 
should  deprive  ourselves  of  credit  card  acquisition  when  we  can  use  it  to  good  effect.) 

Note  that  many  of  these  categories  have  cycle  times  significantly  shorter  than  the  cus¬ 
tomary  seven-  to  ten-year  system  acquisition  cycle.  This  means  that  PMs  must  provide  for 
technology  insertion  during  system  development,  as  well  as  over  the  system’s  service  life. 

Complex  systems  involve  subsystems  and  components  that  draw  from  many 
categories  of  technology.  We  need  processes  and  system  designs  that  provide  the  ability  to 
change  out  obsolete  elements  to  maintain  system-level  performance  without  undue  cost  or 
system  down-time.  Obsolescence  theory  would  help  us  better  understand  and  predict  when 
subsystems  and  components  have  reached  this  level.  QinetiQ  is  beginning  to  do  this  in  the 
United  Kingdom.  We  should  explore  QinetiQ’s  approach  to  developing  better  replacement 
strategies  to  counter  obsolescence. 
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Findings: 


Many  Factors  Affect  Technology 
Acquisition 


GAO  concludes  that  maturing  technology  prior  to  acquisition 
correlated  with  success 

Programs  lack  consistent  process  to  find  technology  opportunities 
PMs  and  industry  are  not  typically  or  consistently  incentivized  to 
find  and  transition  technology 

Parallel  testing  against  mission  capabilities  not  commonly  practiced 
Best  design  practices  facilitate  rapid  technology  acquisition 

-  Modularity  &  partitioning  designs 

-  Open  systems 

-  Open  software 

-  Modeling  &  simulation 

-  Middleware 

-  Application  program  interfaces 


In  1999  a  General  Accounting  Office  study  found  a  strong  correlation  between  tech¬ 
nological  maturity  (at  the  point  of  a  technology’s  insertion)  and  the  success  of  acquisition 
programs.”  Having  achieved  a  high  TRL  is  certainly  not  a  guarantee  of  program  success,  but 
the  correlation  is  too  marked  to  be  ignored. 

The  panel  also  found  that  acquisition  programs  within  the  DON  have  no  consistent 
ability  to  find,  catalogue,  and  assess  technological  opportunities.  This  problem  is  com¬ 
pounded  by  a  general  failure  to  give  PMs  and  contractors  consistent  incentives  to  enhance 
their  programs  with  new  technologies.  We  note  that  such  incentives  are  available,  but  we 
find  that  they  are  not  being  consistently  applied.  And  lastly  there  is  a  growing  frustration  on 
the  part  of  program  personnel  and  financial  officials  with  the  time  and  expense  of  apparently 
duplicative  testing.  They  feel  such  testing  substantially  delays  the  introduction  of  new  tech¬ 
nologies. 

The  panel  found  a  variety  of  design  practices  that  appear  to  facilitate  rapid  technology 
acquisition.  These  include  modularity  and  partitioning,  open  systems,  open  software,  model¬ 
ing  and  simulation,  middleware,  and  application  program  interfaces. 


2  GAO  Draft  Report  dated  7  June  1999  (GAO  Code  707336)  OSD  Case  1836,  Best  Practices:  Managing  tech¬ 
nology  development  can  improve  weapon  system  outcomes. 


19 


This  page  intentionally  left  blank 


20 


Findings: 


Strengthening  Supplier  Consumer 
Relationship  will  build  Confidence 


S&T 


Strengthen  this  path 


Acquisition 
"►  Community  ' 

(Including  Industry) 


Quick  Reaction  Capabilities 


Use  this  path  prudently 


Technological  innovation  is  usually  the  outcome  of  scientific  inquiry.  Within  the 
DON,  S&T  is  the  responsibility  of  a  distinct  community  within  the  research  and  development 
establishment.  The  Panel  believes  that  the  S&T  community  has  not,  despite  its  efforts,  cor¬ 
rectly  identified  its  principal  customer.  While  the  warfighter  is  the  ultimate  user  of  the  prod¬ 
uct,  science  and  technology  normally  finds  its  way  to  the  warfighter  through  some  acquisi¬ 
tion  program.  Paradoxically,  the  S&T  community’s  focus  on  the  operating  forces  may  actu¬ 
ally  be  retarding  development  of  the  tools  the  Fleet  and  Force  need. 

In  general,  the  acquisition  community  lacks  full  confidence  that  the  S&T  community 
is  meeting  its  needs  by  developing  the  right  technology  to  the  level  of  maturity  necessary  for 
insertion  into  programs  of  record.  The  warfighters  see  a  mixed  picture.  Ships  are  often  deliv¬ 
ered  with  out-of-date  capabilities.  Yet  efforts  to  field  innovative  systems  and  capabilities 
rapidly  are  sometimes  poorly  thought-out,  particularly  with  respect  to  their  support  once 
fielded,  instead  of  providing  capable,  reliable,  technologically  advanced  equipment,  the  user 
ends  up  with  “magic  junk.” 

The  Panel  argues  therefore  that  the  principal  customer  of  a  S&T  program  should  be 
an  acquisition  program.  We  think  that  this  will  tend  to  shorten  the  normal  path  from  S&T  to 
fielded  capabilities.  The  occasions  when  the  warfighter  is  a  direct  customer  are  the  excep¬ 
tion — normally,  an  operational  commander  gets  technology  through  the  acquisition  process. 
So  while  there  should  be  paths  directly  between  S&T  and  the  Fleet  and  Force,  these  should 
be  used  sparingly. 

In  special  cases  Advanced  Concept  Technology  Demonstrations,  (ACTDs),  for  ex¬ 
ample)  we  bypass  the  normal  acquisition  process.  ACTDs  place  innovative  new  technolo¬ 
gies  in  the  hands  of  an  operational  sponsor,  permit  them  to  be  evaluated  in  a  realistic  opera- 


21 


tional  environment,  and  leave  residuals  in  the  field  with  the  operating  forces  (unfortunately 
sometimes  without  adequate  logistical  support).  There  are  also  situations  in  which  com¬ 
manders  recognize  emergent  needs  and  communicate  these  to  the  S&T  community  (through, 
for  example,  their  Naval  Research  Science  Advisors,  or  a  Marine  Corps’  Urgent  Needs  Re¬ 
quest).  Although  these  exceptional  pathways  should  be  used  judiciously,  they  nonetheless 
remain  valuable  routes  through  which  new  technologies  reach  the  Fleet  and  the  Force. 

In  sum,  the  science  and  technology  community  should  regard  the  acquisition  commu¬ 
nity  as  its  primary  customer.  At  TRL  3  and  above,  S&T  projects  should  have  an  identified 
acquisition  customer  who  not  only  wants  the  project,  but  is  willing  to  work  in  partnership  to 
mature  the  technology.  We  think  this  customer  should  normally  be  a  PM.  The  acquisition 
community  must  understand  that  the  S&T  community  is  its  technology  supplier,  and  that  the 
warfighter  is  in  turn  its  ultimate  customer.  The  warfighter  needs  up-to-date  systems  that 
work,  and  that  our  Sailors  and  Marines  can  use.  This  PaneFs  recommendations  are  aimed  at 
strengthening  these  important  links. 
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Conclusion: 


Technology  Acquisition: 
No  Single  Path  to  Solution 


Recognize  at  the  outset  that  there  need  be  no  single  path  to  a  solution.  The  panel  be¬ 
lieves  that  in  fact  there  are  multiple  ways  of  closing  the  “Valley  of  Death.”  Just  as  commu¬ 
nication  networks  route  traffic  opportunistically  through  various  paths,  just  as  roadways  af¬ 
ford  multiple  routes  to  a  destination,  so  too  technology  can  move  in  many  ways  from  the 
laboratory  to  a  fielded  system. 

The  Commercial  Technology  Transition  Office  (CTTO)  in  the  Office  of  Naval  Re¬ 
search  moves  certain  technologies  across  the  Valley  (and  its  “deals” — and  the  methods  used 
to  broker  them — merit  greater  emphasis).  FNCs  also  represent  a  good  start  towards  bringing 
valuable  technologies  into  the  acquisition  system. 

The  1999  General  Accounting  Office  study  that  found  a  strong  correlation  between 
TRL  and  acquisition  program  success  recommended  that  S&T  programs  reach  TRL  6  before 
transition  to  acquisition.  The  Department  of  Defense  (DoD)  concurred  with  this  recommen¬ 
dation,  but  has  not  fully  implemented  it.  The  experience  of  the  FNCs  indicates  that  most 
Technology  Transition  Agreeements  (TTAs)  are  negotiated  so  that  the  S&T  community 
brings  its  products  to  TRL  6,  at  which  point  the  acquisition  community  picks  them  up. 

We  think  that  proper  attention  to  TRLs,  and  a  closer  coupling  of  the  S&T  community 
with  the  acquisition  community,  form  the  basis  for  multiple  paths  to  transition.  We  will  re¬ 
port  our  recommendations  in  detail  below,  but  in  brief  we  think  the  four  principal  ways  to 
close  the  Valley  of  Death  are: 

•  CTTO++  (with  emphasis  on  making  more  deals) 

•  FNCs++  (with  tighter  TTAs,  and  fewer  projects  to  better  achieve  a 

critical  mass  of  funding) 
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TRLs  3  and  4. 


Closer  coupling  of  all  S&T  programs  with  acquisition  programs  at 


Rapid  Technology  Acquisition  Teams. 
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Meeting  the  Technology  Acquisition  Challenge 


Requires: 

•  Multiple  Acquisition  Methods 
•S  Traditional 
S  Rapid 


Evolutionary  Spiral  Development 


We  propose  several  changes  in  the  current  acquisition  process  to  address  the  issues 
raised  earlier.  The  first  relates  to  the  creation  of  Rapid  Technology  Acquisition  Teams 
(RTAT)  to  provide  an  alternative,  fast,  design-build-deploy  process.  Various  relatively 
small,  sometimes  ad  hoc,  programs  do  that  now  (ACTDs,  the  Naval  Fleet-Force  Technology 
Innovation  Office,  for  example),  but  we  propose  an  over-arching,  institutionalized  Naval 
process  to  capture  those  exceptional  but  crucial  occasions  when  a  breakthrough  or  an  emer¬ 
gent  need  offers  an  opportunity  for  fast,  effective  technological  support  of  the  warfighter. 
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Meeting  the  Technology  Acquisition  Challenge 


Requires: 

•  Multiple  Acquisition  Methods 

v'  Traditional 
■S  Rapid 

•  Mature  Technology 

■S  Achieve  TRL  6  before  hand-over 


Technology  Readiness  Levels  (TRLs) 


S|Khi  Twt  Liundi 
lOpwibwii 


r\ 

TRL  9 


Actual  system  “flight  proven’  through  successful 
mission  operations 


Actual  system  completed  and  “flight  quafified"  through 
test  and  demonstration 

System  prototype  demonstration  in  a  operational 
environment 


Technology 
Demon  stnbon 


TRL  6 


System/subsystem  model  or  prototype  demonstration 
in  a  relevant  environment 


Component  and'or  breadboard  validation  in  relevant 
environment 


Component  and'or  breadboard  validation  in  laboratory 
environment 

Analytical  and  experimental  critical  function  and/or 
characteristic  proof-of-concept 

Technology  concept  and'or  application  formulated 

Basic  principles  observed  and  reported 

As  Defined  in  5000.2-R 


A  technology’s  maturity  can  be  measured,  and  its  development  managed  and  tracked. 
The  most  accepted  method  uses  TRLs  as  a  principal  tool. 

Tying  S&T  program  decisions  and  management  to  these  TRLs  would  usefully 
inform  decisions  regarding  continuation  or  handover  of  science  and  technology  programs. 
Evidence  also  shows  a  high  correlation  between  program  success  and  the  maturity  of  its 
technologies  before  they  are  incorporated  into  the  program.  TRL  6  has  come  to  be  a  gener¬ 
ally  accepted  level  of  maturity  for  handing  over  a  product  from  science  and  technology  to 
acquisition.  While  DoD  7000. 14-R  places  responsibility  for  achieving  TRL  6  in  Budget  Ac¬ 
tivity  4  (Advanced  Component  Development  and  Prototypes),  we  note  that  the  FNCs  have 
generally  negotiated  TRL  6  as  the  handover  point  from  S&T  to  acquisition. 

The  second  suggested  change  is  to  provide  oversight  of  the  S&T  budget  related  to 
maturing  technology  from  TRL  4  to  TRL  6,  and  thence  into  acquisition  programs.  This  is  the 
intent  of  the  FNCs,  but  the  FNCs  appear  too  diluted.  (This  dilution  is  ironic,  since  one  of  the 
main  reasons  for  setting  up  the  FNCs  in  the  first  place  was  to  concentrate  resources  on  cru¬ 
cial  technology  gaps  in  order  to  achieve  the  critical  mass  necessary  for  transition.)  Fixing 
this  will  require  acquisition  to  identity  key  new  technologies,  provide  paths  to  incorporate 
them  into  actual  acquisition  programs,  and  stimulate  some  demand-pull. 

(See  Appendix  B  for  a  definition  of  TRLs.) 
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Meeting  the  Technology  Acquisition  Challenge 


Requires: 

Multiple  Acquisition  Methods 
S  Traditional 
■S  Rapid 

Mature  Technology 

S  Achieve  TRL  6  before  hand-over 

Incentives  for  Technology  Insertion 


rniMdto 

United  States  Marine  Corps 
Department  of  Navy 

Advanced  Amphibious  Assault  Vehicle  (AAAV'I 
MARK  46  Weapon  System  Development  Team 

In  recognition  ofAeyuintion  Excellence  amt 
Superior  Performance.  The  Mari  46  Weapons  Syxtem 
i»  a  uwU-(/«i  mo, tel  of  aeguiiitioa  reform 
exemplifying  the  principle*  of  CAW.  Interoperabilhty, 
and  I  Pi'll  to  impel, t-e  warfighter  capability  while 
dgnificantly  reducing  total  ownenhtp  coat. 


m 


y 


People  and  organizations  respond  to  incentives.  Properly  structured,  incentives  are  a 
powerful  means  to  create  desired  behaviors. 

PMs  tend  to  be  risk  averse — the  present  system  tends  to  punish  risk-taking — and  new 
technology  inevitably  carries  risk — especially  if  it  is  inserted  prematurely.  The  acquisition 
community  must  receive  incentives  to  accept  reasonable  risk  to  reap  the  cost,  schedule,  or 
performance  benefits  of  technology  insertion.  A  wide  variety  of  contractual  and  personnel 
incentives  exist,  but  have  not  been  consistently  applied  to  this  end.  The  S&T  community 
would  also  benefit  from  metrics — like  well  structured  TTAs  requiring  PM  or  PEO  signature, 
and  properly  applied  TRLs — that  would  give  them  an  incentive  to  seek  out  opportunities  for 
technology  insertion. 
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Meeting  the  Technology  Acquisition  Challenge 


Requires: 

Multiple  Acquisition  Methods 
S  Traditional 
■S  Rapid 

Mature  Technology 

Achieve  TRL  6  before  hand-over 
Incentives  for  Technology  Insertion 
Flexible  T&E  Process 


The  present  test  and  evaluation  process  entails  initial  testing  by  the  contractor,  fol¬ 
lowed  by  developmental  testing  under  the  PM,  and  concludes  with  an  independent  opera¬ 
tional  evaluation  conducted  under  the  guidance  of  the  DoD.  For  the  most  part  these  tests  are 
done  in  series,  with  considerable  duplication.  Very  little  is  done  concurrently  to  reduce  re¬ 
dundancy,  shorten  cycle  time,  and  hold  down  costs.  Considerably  more  needs  to  be  done  in 
this  area.  Refining  and  streamlining  all  test  and  evaluation  could  significantly  speed  technol¬ 
ogy  to  acquisition  and  thus  delivery  to  the  warfighter. 

We  are  convinced  that  a  better-coordinated  test  and  evaluation  process  will  decrease 
the  time  it  takes  to  acquire  new  technology.  A  detailed  treatment  of  test  and  evaluation  is 
beyond  the  scope  of  the  present  study,  but  we  strongly  suspect,  on  the  basis  of  what  we  have 
learned,  that  a  more  flexible  test  and  evaluation  process  would  pay  great  dividends  in  swifter 
insertion  of  new  technology.  We  believe  this  merits  further  study. 
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Meeting  the  Technology  Acquisition  Challenge 


Requires: 

Multiple  Acquisition  Methods 

V  Traditional 

V  Rapid 

Mature  Technology 

■S  Achieve  TRL  6  before  hand-over 
Incentives  for  Technology  Insertion 
Flexible  T&E  Process 

Visibility  of  Available  Technology 


Finally,  the  acquisition  community,  the  warfighter,  and  indeed  the  S&T  community 
itself  need  better  awareness  of  available  technology.  The  Office  of  Naval  Research  Global 
and  the  CTTO  both  provide  some  of  this  awareness,  but  a  comprehensive,  accessible  knowl¬ 
edge  base  of  Naval  S&T  (including  work  from  other  Defense  organizations  with  Naval  ap¬ 
plications,  other  governmental  agencies,  commercial  sources,  and  international  entities)  is 
still  needed. 
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Meeting  the  Technology  Acquisition  Challenge 


Requires; 

•  Multiple  Acquisition  Methods 

■S  Traditional 
S  Rapid 

•  Mature  Technology 

V  Achieve  TRL  6  before  hand-over 

•  Incentives  for  Technology  Insertion 

•  Flexible  T&E  Process 

•  Visibility  of  Available  Technology 


Yields: 

•  Up  to  Date  Military  Capability 

•  Reduced  Risk  of  Technology  Insertion 

•  Rapid  Response  When  Needed 
•Technologically  Current  Workforce 


Here  is  the  upshot  of  our  recommendations,  if  they  are  implemented.  The  goal  is  a 
Navy-Marine  Corps  Team  with  the  most  advanced,  the  most  agile,  and  the  most  affordable 
technological  capabilities  possible.  We  can  achieve  this  by  reducing  the  risk  of  inserting 
technology  into  acquisition  programs,  by  providing  rapid  response  when  needed,  and  by 
maintaining  situational  awareness  of  technological  possibilities  world-wide. 
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Findings  &  Rapid  Response  Needed 

Conclusions: 


•  Certain  high  priority  external  demands  require  a  shorter  S&T 
response  cycle 

•  Technology  base  provides  unforeseen/unplanned  opportunities 

•  Did  not  find  an  institutionalized  rapid  acquisition  process  across 
DON 

-  Did  find  successful  USMC  example  in  their  Urgent  Needs  Request 
process 


•  Solution  space  must  be  bound  by  time,  complexity,  funds 

•  Naval  Research  Science  Advisors  provide  ONR  link  to  Fleet/Force 

•  Program  teams  need  to  be  integrated  with 
SYSCOMs/PEOs/DRPMs  to  include  engineering  and  logistical 

systems 


The  FNC  process  and  its  associated  time  lines  have  come  to  form  the  baseline  for 
technology  development  and  transition  within  the  DON.  However,  it  is  also  clear  that  no  sin¬ 
gle  process  or  methodology  for  technology  transition  can  apply  to  every  circumstance  or 
meet  every  need.  This  is  particularly  true  for  Naval  forces,  which,  inherently  expeditionary 
as  they  are,  face  shifting  and  complex  threats. 

A  parallel,  streamlined  process  that  emphasizes  a  much  shorter  response  cycle  to  sup¬ 
port  selected  high  priority  needs  of  the  operating  forces  is  necessary.  These  “out-of-cycle” 
high-priority  needs  often  emerge  in  response  to  pre-deployment  or  contingency  work-ups, 
and  imminent  or  on-going  combat  operations.  Such  a  rapid  response  capability  would  also 
foster  unforeseen  and  unplanned  capitalization  of  the  technology  base  as  the  result  of  “rapid 
insight”  opportunities,  associated  with  the  analysis  of  asymmetrical  threats  and  the  need  for 
non- traditional  counters. 


The  DON  does  not  now  have  a  fonnal,  routine,  documented  method  of  consistently 
addressing  high-priority  demands  for  rapid  technology  acquisition  (even  though  history  of¬ 
fers  many  examples  of  this  sort  of  process  occurring  on  an  emergency  or  ad  hoc  basis — the 
DON’s  participation  in  Team  Tango  is  a  recent  example). 

The  Marine  Corps  does,  however,  have  a  documented  and  functioning  Urgent  Needs 
Request  process  that  has  worked  successfully  for  several  years  and  could  be  adapted  or  form 
the  basis  for  a  Department-wide  method. 

There  are  three  keys  to  establishing  an  effective  rapid  response  capability: 
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•  Define  the  boundaries  of  a  high-priority  initiative.  Such  an  initiative  will  gen¬ 
erally  have  a  limited  response  time,  a  short  required  time-to-market,  relatively  simple  as¬ 
sociated  technologies  to  be  transferred,  and  a  small  to  moderate  level  of  investment. 

•  Establish  clear  and  regular  communication  among  the  principal  customers 
(operating  forces  and  the  acquisition  community)  and  the  primary  suppliers  (the  Office  of 
Naval  Research  and  the  Deputy  Assistant  Secretary  of  the  Navy  (Research,  Development, 
Test,  and  Evaluation)).  Such  linkages  are  already  partially  in  place  with  the  Naval  Re¬ 
search  Science  Advisors  at  all  major  Fleet  and  Force  commands.  Technology  liaison  po¬ 
sitions  in  the  acquisition  community  would  make  a  major  contribution  and  fill  a  large 

gap- 

•  Insure  that  teams  formed  to  execute  rapid  response  requests  are  associated 
with  the  appropriate  SYSCOM  or  PEO.  This  is  particularly  important  to  insure  that 
proper  system-wide  safety  and  assurance  needs  are  met,  and  that  future  logistical  life  cy¬ 
cle  support  issues  are  resolved  (in  order  to  avoid  the  “magic  junk”  syndrome  common  in 
the  past). 
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Recommendations: 


Create  Rapid  Response  Capability 


•  Establish  a  rapid  response  process 

-  Review  OSD  “Team  Tango”  and  USMC  Urgent  Needs  Request 
processes  for  applicability  and  lessons  learned 

•  Create  Rapid  Technology  Acquisition  Teams  (RTAT) 

•  Use  PEO/PM’s  CTO  or  “Technology  Liaison”  as  linkage 
between  ONR  and  acquisition 

•  Assign  the  responsibility  for  this  process  to 
DASN(RDT&E) 


The  Panel  recommends  that  the  Department  establish  a  fonnal  rapid  response  tech¬ 
nology  acquisition  process  and  create  short-tenured  RTAT  to  accomplish  assigned  tasks. 
These  interdisciplinary  teams  would  be  assembled  from  both  the  S&T  and  acquisition  com¬ 
munities,  and  would  exist  only  as  long  as  the  task  remained.  The  Marines’  Urgent  Needs  Re¬ 
quest  provides  a  model  of  how  the  DON  might  initiate  a  RTAT. 

The  Panel  further  recommends  that  SYSCOMs,  PEOs,  and  DRPMs  use  their  Chief 
Technology  Officers  (CTOs)  as  the  link  between  the  Office  of  Naval  Research  and  the  acqui¬ 
sition  community.  For  organizations  that  do  not  have  CTOs  or  other  “tech-finders,”  billets 
should  be  established,  immediately  staffed,  and  assigned  this  and  other  related  responsibili¬ 
ties.  The  Naval  Research  Science  Advisors  who  play  an  analogous  role  on  operational  staffs 
may  provide  a  model  of  such  linkage.  Responsibility  for  establishing  and  funding  this  new 
capability  should  probably  rest  with  the  Deputy  Assistant  Secretary  of  the  Navy  (Research, 
Development,  Test  and  Evaluation). 
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Findings  &  Technology  Maturity  is  Key 

Conclusions: 


•  There  is  a  high  correlation  between  technology  maturity 
(TRLs)  at  insertion  and  program  success 

•  Managing  technology  to  maturity  is  improved  by: 

-  Identified  “consumers” 

-  Formal  agreements 

-  Technology  management  more  tightly  aligned  with  acquisition 
program 

•  FNC  process  has  increased  likelihood  of  developing 
technology  to  acceptable  TRL 

•  Disruptive  technologies,  which  may  provide  leap  ahead 
capabilities,  threaten  programs  of  record 


The  experience  of  the  Panel  as  well  as  the  General  Accounting  Office’s  conclusions 
strongly  suggest  that  achieving  a  high  level  of  maturity  before  new  technologies  are  inserted 
into  product  development  significantly  increases  the  probability  of  program  success.  High 
levels  of  technical  maturity  obviously  cannot  insure  the  success  of  a  program  but  attempting 
to  adopt  technology  before  high  levels  of  maturity  have  been  reached  substantially  increases 
risk  to  a  program’s  cost  and  schedule.  As  the  General  Accounting  Office  notes  in  its  report 
on  the  impact  of  technological  maturity  on  program  success,  “It  is  a  rare  program  that  can 
proceed  with  a  gap  between  product  requirements  and  the  maturity  of  key  technologies  and 
still  be  delivered  on  time  and  within  costs.” 

The  Navy’s  FNC  S&T  management  and  alignment  process  holds  promise  for  ad¬ 
dressing  the  challenge.  Important  technologies  are  being  matured,  and  formal  agreements 
between  the  acquisition  community  and  the  S&T  community  are  beginning  to  foster  technol¬ 
ogy  transition.  Here  are  some  of  the  reasons  the  FNC  process  is  showing  promise: 

•  It  is  aligned  with  future  capability  needs  and  helps  to  close  gaps  be¬ 
tween  the  “as  is”  situation  and  the  future  capability  requirement, 

•  The  FNCs  are  reviewed  periodically  and  realigned  if  necessary  with 
future  needs, 

•  All  FNC’s  are  led  by  an  Integrated  Product  Team  composed  of  the  ma¬ 
jor  stake  holders, 

•  Formal  technology  transfer  agreements  between  the  S&T,  acquisition, 
and  requirements  communities  are  established  regarding  technology  maturation  and 
program  insertion, 
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•  And,  a  reasonable  goal,  80%  of  all  FNC  developmental  elements,  is 
established  for  requiring  formal  agreements. 

If  the  FNCs  provide  a  model  for  the  acquisition  of  evolutionary  technology,  the  DON 
still  needs  a  way  of  fostering  the  development  and  acquisition  of  disruptive  technologies.3 

Disruptive  technologies  threaten  programs  of  record  but  are  essential  to  future  Naval 
superiority.  The  Naval  acquisition  community  structure  must  provide  a  home  for  their  de¬ 
velopment  without  intimidating  the  established  programs  of  record. 


3  A  disruptive  technology  is  an  innovation  that  leads  to  new  classes  of  products  that  are  cheaper,  better,  and 
more  convenient  than  their  predecessors.  These  products  are  typically  adopted,  initially,  only  by  a  small,  fringe 
set  of  users,  and  rarely  emerge  in  response  to  customer  demand.  Disruptive  technologies  have  features  that  a 
few  fringe  (and  generally  new)  customers  value.  Products  based  on  disruptive  technologies  are  typically 
cheaper  to  produce,  simpler,  smaller,  better  performing,  and,  frequently,  more  convenient  to  use.  A  sustaining 
technology  improves  the  performance  of  established  products.  Sustaining  technologies  are  usually  developed 
by  successful  and  well  established  companies  who  often  hold  a  leadership  position  in  their  industries.  (The 
terms  originated  with  Clayton  Christensen,  Professor  of  Business  Administration  at  Harvard  Business  School.) 
In  this  context,  the  FNCs,  as  evolutionary  programs,  are  generally  in  the  business  of  developing  sustaining  as 
opposed  to  disruptive  technologies. 


42 


Recommendations: 


Increase  Acquisition  Community 
Confidence  in  Technology  Maturity 


Further  improve  technology  maturation  process: 

-  Require  Acquisition  Advocate  at  TRL  3 

-  Require  Technology  Transfer  Agreement  (TTA)  at  TRL  4 

-  Hand-over  S&T  program  oversight  &  approval  to  acquisition 
program  at  TRL  4 

-  Review  TTAs  annually  to  ensure  alignment 
Establish  technology  liaison  position/function  in 
acquisition  programs  where  they  do  not  exist 

Create  Direct  Reporting  Program  Office  to  manage  leap 
ahead  technology  to  acceptable  level  of  maturity 


The  Panel  found  that  the  FNC  process  could  be  improved  by  further  expanding  the 
formal  agreement  process,  and  by  passing  oversight  of  the  S&T  program  to  the  acquisition 
community  at  the  appropriate  time.  Presently  the  requirements  community  is  the  dominant 
partner  in  the  FNCs,  we  think  this  role  would  be  better  filled  by  the  acquisition  community. 

To  help  eliminate  the  so-called  “Valley  of  Death,”  the  Panel  recommends  that  80%  of 
the  S&T  programs  that  have  matured  to  TRL  3  (active  research  and  development)  should  be 
required  to  have  a  formally  acknowledged  advocate  in  the  acquisition  community  who  needs 
the  technology  and  endorses  further  development  and  maturation.  In  addition,  80%  of  all 
S&T  programs  should  be  required  to  have  a  signed  TTA  in  place  by  TRL  4.  Furthermore,  if 
an  acquisition  program  assumed  oversight  of  a  S&T  program  at  TRL  4  (when  components  or 
breadboards  are  validated  in  a  laboratory  environment),  the  acquisition  program  would  gain 
intimate  knowledge  of  the  technology’s  development  status,  rate  of  maturing,  and  risk  of  in¬ 
sertion.  To  this  end,  we  believe  that  acquisition  community  PMs  should  provide  manage¬ 
ment  oversight  to  S&T  programs  from  TRL4  through  formal  transition  into  acquisition  pro¬ 
grams.  When  a  technology  applies  to  more  than  one  program,  a  single  PM  should  be  given 
the  lead.  We  recommend  that  the  S&T  programs’  TTAs  be  reviewed  annually,  with  progress 
toward  higher  TRLs  and  alignment  with  evolving  requirements  receiving  particular  attention. 

The  accelerating  pace  of  technological  change,  the  inherent  complexities  of  these 
technologies,  and  the  multitude  of  their  sources  of  origin  are  well  beyond  the  ability  of  any 
single  acquisition  PM  to  assimilate  and  act  on.  In  view  of  this,  and  with  the  absolute  need  to 
connect  acquisition  programs  of  record  with  Naval  S&T,  the  Panel  strongly  believes  that 
dedicated  “technology  finders”  or  liaison  personnel  must  be  established  to  create  the  critical 
bridge  to  technology  hand-over.  These  acquisition  technologists  in  the  program  offices 
would  have  insight  into  the  problems  and  opportunities  of  each  program,  and  help  search  for 
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the  best  science  and  technology  solutions  available.  These  personnel  could  also  manage  the 
related  portfolio  of  science-and-technology-like  activities  present  in  every  program — like 
Small  Business  Innovation  Research  (SBIR)  and  Foreign  Comparative  Test  (FCT)  programs. 
For  the  Systems  Commands,  an  individual  technology  liaison  position  could  administer  a 
basket  of  smaller,  related  programs  within  a  division  or  product  group.  They  would  also  be 
the  principal  user  of  the  Panel’s  recommendation  below  dealing  with  the  creation  of  a  tech¬ 
nology  knowledge  base. 

The  Panel  also  notes  that  “disruptive  technology”  must  be  accommodated  and  man¬ 
aged  by  the  S&T  and  acquisition  communites.  These  technologies,  as  opposed  to  “sustaining 
technology,”  usually  pose  too  high  a  risk  to  an  established  program  of  record.  (They  fre¬ 
quently  become  available  at  what  is  usually  an  unacceptably  low  TRL,  say  TRL  3  or  4.) 
However,  disruptive  technologies  offer  potentially  transformational  capabilities.  We  think 
they  should  be  incubated  in  some  specialized  process.  The  Assistant  Secretary  of  the  Navy 
(Research,  Development,  and  Acquisition)  should  establish  a  Direct  Reporting  Program 
Manager  (DRPM)  charged  with  maturing  these  disruptive  technologies  to  lower  levels  of  risk 
so  an  acquisition  program  can  adopt  them.  This  advocate  for  disruptive  technologies  would 
be  charged  with  taking  more  risk  and  so  will  have  certain  S&T  programs  fall  by  the  wayside 
as  they  fail  to  prove  useful.  But  when  such  programs  fail,  they  would  not  jeopardize  any  par¬ 
ticular  program  of  record.  Without  an  office  of  this  type,  higher  risk  but  potentially  impor¬ 
tant  technologies  may  not  be  developed  because  of  the  threat  they  pose  to  established  pro¬ 
grams’  schedules  and  budgets. 
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Findings: 


Proper  use  of  Metrics  and  Incentives 
Improves  Results 


•  Metrics  inspected  and  incentivized  correlate  directly 
with  accomplishment 

•  Existing  incentives  have  not  been  consistently  applied  to 
technology  insertion 

•  Innovative  acquisition  strategies  can  incentivize  PMs 
and  industry  to  increase  technology  insertion 


Organizations  do  well  what  the 
boss  checks  and  rewards 


y 


In  both  government  and  industry,  what  you  inspect  and  incentivize  is  what  you  ac¬ 
complish.  The  acquisition  reform  movement  has  exhaustively  scrutinized  contractual  and 
personnel  incentives  in  general.  A  wide  variety  of  incentives  exists,  but  they  generally  have 
not  been  consistently  applied  to  promote  technology  acquisition. 

The  introduction  of  new  technology  always  carries  with  it  risks  to  cost,  schedule  and 
performance.  PMs  are  understandably  risk-averse  and  so  are  generally  reluctant  to  be  early 
adopters  of  a  new  technology. 

The  motivation  to  introduce  new  technology  is  generally  either  to  improve  perform¬ 
ance  or  reduce  cost — normally  acquisition  cost,  sometimes  life  cycle  cost.  For  acquisition 
cost,  if  profit  is  tied  to  percentage  of  the  cost  of  the  contract,  the  contractor  has  no  incentive 
to  reduce  cost  by  technology  insertion  unless  the  scope  and  resources  of  the  base  contract  are 
expanded.  Similarly,  the  PM  is  typically  not  motivated  to  reduce  program  cost  because,  if  the 
program  actually  recovers  savings,  the  comptroller  normally  sweeps  them  up  to  fund  short¬ 
falls  in  other  programs. 

Life  cycle  cost  (LCC)  or  total  ownership  cost  (TOC)  represents  the  sum  of  acquisition 
cost  plus  operations  and  support  (O&S)  costs  over  the  system’s  service  life.  Annual  O&S 
cost  targets  are  being  introduced  in  recent  Operational  Requirements  Documents  (ORD) 
(now  called  “Capability  Development  Documents”  (CDD))  for  system  acquisitions  once  the 
initial  cost  versus  capabilities  trade-offs  have  been  determined.  This  alone  would  provide  an 
incentive  to  insert  technologies  in  order  to  meet  stated  O&S  cost  requirements.  (Adopting 
technologies  that  would  reduce  manning  requirements  is  one  good  example.)  Once  the  tech¬ 
nology  required  to  achieve  LCC  targets  in  the  ORD  or  CDD  has  been  identified,  PMs  and 
contractors  have  little  incentive  to  further  reduce  LCC  through  further  technology  insertion, 
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since  this  would  almost  certainly  increase  acquisition  cost.  The  situation  regarding  perform¬ 
ance  is  similar.  Technology  insertion  may  be  a  way  of  meeting  the  performance  require¬ 
ments  stated  in  the  ORD  or  CDD,  but  acquisition  cost  limits  constrain  the  PMs  and  contrac¬ 
tors  from  going  further. 

Innovative  competitive  acquisition  strategies  and  contract  clauses  can  give  PMs  and 
industry  incentives  to  increase  technology  acquisition.  Recent  examples  include  the  Acoustic 
Rapid  Commercial-off-the-shelf  Insertion  (ARCI)  program,  DD21  (now  DD(X)),  and  Joint 
Strike  Fighter  (JSF)  programs.  The  goal  of  the  submarine  ARCI  program  is  to  improve  the 
U.S.  Submarine  Force’s  ability  to  detect  hostile  submarines  at  greater  ranges.  This  program  is 
using  open  software  architectures  and  innovative  competitive  procurement  strategies.  The 
DD(X)  and  JSF  programs  have  developed  some  very  innovative  contractual  approaches  that 
offer  incentives  to  both  PMs  and  contractors  to  insert  technology.4 

Metrics  can  be  used  to  enhance  the  transfer  of  new  technology  from  the  S&T  com¬ 
munity  to  the  acquisition  community  if  criteria  are  defined  and  the  results  reported  and 
tracked.  Today,  there  is  no  requirement  that  S&T  projects  below  TRL  4  have  advocates  in 
the  acquisition  community.  Currently,  TTAs  are  signed  between  the  S&T  community  and  the 
acquisition  community  for  80%  of  the  projects  within  each  FNC  program  when  these  pro¬ 
jects  reach  TRL  4.  There  is  no  requirement  for  such  agreements  in  exploitation  and  deploy¬ 
ment  projects  outside  the  FNCs. 


4  Naval  Research  Advisory  Committee  Report,  Life  Cycle  Technology’  Insertion  (July  2002). 
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Recommendations: 


Employ  Metrics  and  Incentives  to 
Drive  Behavior  and  Culture  Change 


Insure  that  each  S&T  project  has  an  acquisition  advocate  at  TRL  3 

-  Cancel  projects  that  don’t 

Insure  that  S&T  projects  have  a  signed  TTA  at  TRL  4 

-  Cancel  projects  that  don’t 

Require  that  acquisition  strategy  and  acquisition  plans  identify 
technology  transfer  incentives  and  corresponding  metrics 
Require  that  a  technology  insertion  plan  be  criterion  for  contract 
award 

MDAs  establish  an  exit  criterion  which  identifies  the  TTA 
accomplishments  required  to  proceed  to  the  next  program  phase 
Require  that  both  military  and  civilian  performance  evaluations 
include  success  in  technology  acquisition/insertion 


The  Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition)  should 
direct  that  metrics  be  established  to  promote  technology  acquisition.  Each  S&T  project, 
whether  or  not  it  directly  supports  an  FNC,  should  be  required  to  identify  an  acquisition  ad¬ 
vocate — an  acquisition  PM — when  it  reaches  TRL  3.  Projects  not  endorsed  by  the  acquisition 
community  at  TRL  3  should  be  cancelled,  except  in  extraordinary  circumstances.  Addition¬ 
ally,  80%  of  all  S&T  projects  must  be  required  to  have  a  signed  TTA  at  TRL  4.  Currently, 
only  projects  supporting  a  FNC  are  required  to  do  this.  The  S&T  portfolio  should  be  re¬ 
viewed  annually  to  insure  that  its  programs  are  meeting  this  guidance.  If  this  practice  were 
adopted  across  the  exploitation  and  deployment  portion  of  the  Naval  science  and  technology 
portfolio,  leaders  in  the  DON  would  have  the  information  they  needed  to  take  action  if  pro¬ 
gram  goals  were  not  being  met. 

Both  contractual  and  personnel  incentives  are  required  to  temper  the  risk- averse  PM 
and  contractor  culture  and  enhance  technology  acquisition.  An  example  of  contractual  incen¬ 
tives  is  the  “gain  sharing”  strategy  in  which  a  portion  of  the  acquisition  cost  savings  resulting 
from  technology  insertion  are  retained  and  shared  by  both  the  program  office  and  the  con¬ 
tractors.  In  this  way,  as  actual  savings  occur,  part  of  them  could  be  retained  by  the  program 
office  and  used  to  address  other  opportunity  areas.  The  contractor  could  also  retain  their 
original  negotiated  profit  plus  a  share  of  the  savings  generated  for  a  net  profit  gain.  Contrac¬ 
tual  incentives  such  as  this  would  encourage  managed  risk-taking.  The  Panel  recommends 
the  following  specific  actions: 

•  At  program  initiation,  prior  to  initial  contract  award,  require  that  the  Program 
Strategy  and  Acquisition  Plan  (AP)  specifically  identify  technology  acquisition  incen¬ 
tives  and  the  corresponding  behaviors  and  results  expected. 


47 


•  Require  that  technology  acquisition  and  insertion  be  an  evaluation  or  source 
selection  criterion. 

•  Require  that  Milestone  Decision  Authorities  (MDA)  establish  an  exit  criterion 
at  each  formal  milestone  review  that  identifies  the  expected  TTA  accomplishments  dur¬ 
ing  the  next  development  phase  of  a  program  and  include  that  achievement  as  a  prerequi¬ 
site  for  completing  the  phase  being  entered. 

The  DON’s  military  and  civilian  personnel  management  systems  can  also  be  used  to 
enhance  technology  acquisition.  The  performance  evaluation  system  as  it  stands  now  as¬ 
sesses  performance  of  duty.  When  it  is  understood  that  technology  acquisition  is  one  of  Pro¬ 
gram  Officer’s  or  PM’s  duties,  it  should  be  possible  to  reflect  performance  of  that  duty  in 
evaluation  reports. 
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Findings  & 
Conclusions: 


>k 


T&E  Could  Yield  Large  Dividends  in 
Reducing  Technology  Acquisition  Time 


K. 


•  Testing  tends  to  be  serial,  redundant,  and  uncoordinated 
(Contractor,  Development,  Operational) 

•  T&E  is  focused  more  on  rigid  performance 
specifications  than  mission  capabilities 

•  Spiral  development  suggests  the  need  for  alternative 
testing  approaches 

•  Appropriate  integration  of  modeling  &  simulation  with 
testing  could  significantly  reduce  and  focus  test 
requirements 

Panel  received  widely  different  views  of  T&E 
problems/solutions  -  but  did  not  have  time  to  conduct  a 
comprehensive  assessment 
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OPNAV  (N76)  has  expressed  concern  that  33%  of  every  R&D  dollar  is  spent  on  test 
and  evaluation,  and  that  this  fraction  is  too  high.  It  also  appears  that  the  overall  testing  cycle 
from  contractor  testing  through  the  PM’s  developmental  testing  and  concluding  with  Director 
of  Test  and  Evaluation  (DoT&E)  operational  testing,  is  too  long,  redundant,  and  poorly  coor¬ 
dinated.  Furthennore,  most  testing  and  evaluation  are  conducted  in  a  serial  fashion,  where 
each  test  starts  from  the  beginning  and  repeats  much  of  what  has  already  been  accomplished. 

The  Panel  heard  many  different  views  on  the  advantages  and  disadvantages  of 
current  processes  and  methods,  but  there  was  general  agreement  that  the  present  approach  is 
costly  and  extends  the  time  it  takes  to  transition  new  technology  to  the  warfighter. 

In  the  area  of  Test  and  Evaluation  Master  Plans  (TEMPs),  there  is  a  strong  perception 
that  these  are  too  focused  on  specifications.  They  can  be  inflexible  where  they  need  to  adapt 
to  changing  requirements  and  concepts  of  operations  (conops). 

Moreover,  as  the  Navy  and  Marine  Corps  increasingly  embrace  spiral  development, 
this  testing  process  could  become  even  more  redundant,  as  tests  would  presumably  have  to 
be  repeated  in  each  cycle.  Parallel  testing  of  contractor,  PM,  and  DoT&E  requirements  could 
help,  along  with  the  acceptance  of  more  modeling  and  simulation  as  a  method  of  evaluation. 
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Recommendations: 


Capitalize  on  Potential  T&E 
Opportunities 


Task  detailed  study  of  DON  T&E  to  review  process 
efficiency  and  determine  if  there  are  redundancies  that 
can  be  eliminated 

-  Address  findings  &  conclusions 

-  Consider  recommending  OSD  responsibility  for  all  T&E 
oversight  and  certification  be  placed  under  JFCOM  as  the  end 
user/warfighter  to  enable 

•  Joint  Focus 

•  Relevant  CONOPS 


Further  study  is  required  to  determine  if  redundancy  in  test  and  evaluation  can  be  re¬ 
duced  through  parallel  testing.  As  joint  operations  have  become  the  nonn,  and  with  Joint 
Forces  Command  (JFCOM)  now  tasked  with  responsibility  for  all  experimentation,  training, 
exercise  coordination,  interoperability,  force  provisioning,  and  development  of  war  fighting 
policy,  consideration  should  be  given  to  moving  all  Defense-level  test  and  evaluation  under 
the  leadership  of  the  ultimate  customer  of  technology  acquisition  for  better  control  and  focus. 

We  recommend  a  detailed  study  of  Naval  test  and  evaluation  to  find  process  efficien¬ 
cies  and  identify  any  redundancies  that  might  be  eliminated.  In  particular,  the  study  should 
examine  policy  that  would  take  advantage  of  parallel  testing.  As  a  point  of  departure,  such  a 
study  might  also  examine  the  benefits  of  placing  responsibility  for  all  Defense-level  test  and 
evaluation  oversight  under  some  representative  of  the  end  user  and  warfighter.  This  might 
promote  joint  focus  and  more  relevant  conops. 
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Findings  &  Technology  Awareness  Needed 

Conclusions: 


•  Technology  transition  opportunities  not  consistently 
visible  to  acquisition  programs 

-  Government  +  Commercial  =  $298B/year 

-  International  =  $???B/year 

•  Programs  lack  consistent  process  to  identify  technology 
acquisition  opportunities 

•  Accelerating  technology  development  confounds 
knowledge  currency 


It  is  extremely  difficult  for  acquisition  programs  to  know  about  the  wide  array  of 
Naval  S&T  programs.  The  balance  of  Defense-wide  science  and  technology  is  even  less 
visible.  Other  government  agencies — like  the  Department  of  Energy,  NASA,  the  National 
Institutes  of  Health,  the  Department  of  Transportation,  and  the  Department  of  Homeland  Se¬ 
curity — also  develop  technologies  that  may  have  Naval  applications.  It  is  even  more  difficult 
for  acquisition  programs  to  gain  awareness  of  these  technologies.  Commercial  S&T  invest¬ 
ment — including  both  United  States  and  foreign — now  far  outstrips  that  of  the  DoD.  Such 
investment  is  often  proprietary,  or  otherwise  invisible. 

The  problem  of  successful  technology  acquisition  is  further  exacerbated  by  lack  of  in¬ 
stitutionalized  processes  to  identify  technology  transition  opportunities  for  the  acquisition 
community.  We  find,  for  example,  that  there  is  no  comprehensive,  readily  accessible,  user- 
friendly  S&T  data  base.  Thus  a  difficult  situation  gets  worse. 

Many  search  engines  are  available  for  mining  such  a  knowledge  base,  once  it  is  cre¬ 
ated.  For  example,  the  Materials  and  Structures  Branch  in  Naval  Air  Systems  Command  has 
a  comprehensive  database  under  the  aegis  of  the  Aerospace  Materials  Technology  Consor¬ 
tium.  This  is  a  partnership  among  academia,  industry,  and  government  to  describe  all  the 
latest  materials  technologies  and  processes.  The  Department  of  Energy  uses  search  engines 
to  find  and  evaluate  the  potential  of  new  technology.  Such  tools  include  Vx  Insight,  Auregin, 
STARLIGHT,  SPIKE,  VantagePoint,  and  ClearForest. 

While  United  States  investment  in  research  and  development  dwarfs  that  of  allied  and 
friendly  countries,  many  of  these  countries  excel  in  specialized  areas.  Technologically  ad¬ 
vanced  European,  Asian,  and  Latin  American  nations  have  a  substantial  and  growing  com- 
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mercial  research  and  development  base,  and  their  constrained  defense  funding  has  led  them 
to  some  innovative  technology  acquisition  strategies  that  could  be  useful  to  the  DoD. 

Awareness  of,  and  access  to,  technology  from  all  sources,  including  international 
ones,  facilitates  technology  acquisition  for  new  systems  at  any  stage.  The  problem  of  knowl¬ 
edge  currency  is  compounded  by  the  increasing  number  of  innovations  and  technology 
breakthroughs  in  ever-decreasing  time  frames.  Technology  grows  and  finds  its  uses  at  a 
geometric  rate. 
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Recommendations: 


Technology  Transition  Candidate 
Knowledge  Base 


Create  a  knowledge  base  of  technology  development 
activities 

-  Tool  for  acquisition  program  offices  to  identify  transition 
opportunities 

-  Tracks  technology  development  through  TRLs 

-  Attributes  defined  by  PMs 

-  Created  and  periodically  updated  by  ONR 

-  Start  with  ONR,  expand  to  other  DoD,  other  government, 
commercial,  and  foreign  activities 


Data  mining  is  a  powerful  tool  for  identifying  trends  and  opportunities.  The  DON 
should  create  and  maintain  a  searchable  data  base  of  S&T  activities.  We  could  also  use  such 
an  approach  to  track  the  evolution  of  technologies  through  TRLs.  The  customer  (the  acquisi¬ 
tion  community,  and  particularly  the  PMs)  must  define  the  data  base’s  attributes  in  order  to 
insure  its  usefulness.  The  Office  of  Naval  Research  is  the  logical  authority  to  create  and 
maintain  such  a  data  base. 


The  broad  scope  of  technology  development  activities — military,  other  government, 
commercial,  and  international — makes  awareness  of  technology  insertion  opportunities  a 
daunting  responsibility.  If  the  acquisition  community  is  to  recognize  such  opportunities,  it 
will  need  technology  liaison  positions  analogous  to  the  operating  forces’  Naval  Research 
Science  Advisors  and  the  Marine  Corps  Systems  Command  Liaison  Officers.  Such  positions 
should  be  established  within  larger  acquisition  program  offices,  and  within  the  PEOs  and 
SYSCOMs  to  cover  the  smaller  programs.  The  technology  liaison  personnel  would  track 
emerging  technologies  likely  to  prove  relevant  to  each  program  of  record.  Data  mining  tools 
(like  those  created  by  the  Department  of  Energy),  Naval  technology  experts,  ONR  Global, 
and  commercial  technology  surveys  could  be  used  as  resources.  An  effective  technology  li¬ 
aison  as  a  regular  member  of  the  acquisition  team  should  help  programs  not  only  to  avoid 
missing  opportunities  to  insert  valuable  technology,  but  also  to  warn  them  away  from  relying 
on  immature  or  inappropriate  technologies. 


55 


This  page  intentionally  left  blank 


56 


No  New  Rules,  Regulations  or  Policies  Required 


DASN(RDT&E) 


ASN(RDA) 


DAU/PEO 


ASN(RDA) 


DASN  (RDT&E) 


We  believe  that  no  new  rules,  regulations,  or  policies  are  required,  and  we  recom¬ 
mend  that  the  following  offices  be  assigned  responsibility  for  implementation. 
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Increasing  Consumer  Confidence 


/Create  Rapid  TechN- - 

(  Acquisition  Team  \  Address  Rapid  /  Transformational  Tech  Acq 
y  &  Future  Business  J  "~ 

f  Ensure  '  - — 

(  Technology  ) 

/^Measure _  c  Behavior 

A""  More 

Responsive  \ 
To 

Warfighter’s  J 
Npprk 

f  Conduct  T&e'N  AIi8d  Wlth  Evolutionary  Acq _ 

V  Study  J 

/X reate  Techno  logyx 
(  Knowledge  ) 

\.  Base  /  Avoid  Missed  Opportunities 

Implicit  in  Sea  Power  21  is  an  imperative  to  speed  development  of  new  concepts  and 
technologies  and  to  streamline  the  procedures  that  facilitate  their  rapid  implementation.  The 
recommendations  of  this  study  will  shorten  the  time  required  to  respond  to  the  warfighter’s 
needs. 


Institutionalizing  the  concept  of  a  RTAT  would  establish  an  ability  to  respond  rapidly 
to  the  warfighter’s  unique  and  immediate  needs,  bypassing  the  more  deliberate  acquisition 
process.  The  approach  the  Panel  recommends  would  insure  that  logistical  support  is  pro¬ 
vided  and  maintained  for  items  delivered  through  this  accelerated  process.  A  Direct  Report¬ 
ing  Program  Office  for  Disruptive  Technology  would  incubate  promising  technologies  until 
they  are  ready  to  be  handed  over  to  an  established  program.  This  office  would  also  provide  a 
home  for  disruptive  technologies  emerging  from  discovery  and  invention.  Since  these  poten¬ 
tially  transformational  technologies  are  not  linked  to  existing  acquisition  programs,  they  need 
an  alternative  path  to  maturity. 

Requiring  an  S&T  project,  whether  or  not  it  directly  supports  an  FNC,  to  identify  an 
acquisition  advocate  when  it  reaches  TRL  3,  and  to  have  a  signed  TTA  at  TRL  4,  will  facili¬ 
tate  a  more  rapid  maturation  and  acceptance  of  technology.  It  would  focus  resources  on  S&T 
that  have  a  customer,  and  it  would  give  the  customer  responsibility  for  program  oversight 
beyond  TRL  4.  The  net  result  should  be  a  higher  yield  from  the  S&T  program. 

Using  established  metrics  and  TRLs  to  measure  a  technology’s  readiness  for  insertion 
into  acquisition  programs  will  improve  the  chances  of  program  success.  Not  only  will  spe¬ 
cific  programs  benefit,  but  the  consumer’s  confidence  in  S&T  will  also  increase,  to  the  gen¬ 
eral  improvement  of  the  research  and  development  culture.  Contractual  and  personnel  incen¬ 
tives  to  temper  risk  aversion  and  encourage  technology  insertion  should  further  improve  the 
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maturation  and  infusion  of  technology.  The  synergy  of  these  initiatives  will  encourage 
proper  assumption  of  well-managed  risk  to  meet  the  needs  of  the  Fleet  and  Force. 

The  test  and  evaluation  process  is  often  described  as  a  principal  obstacle  to  rapid 
handover  of  technology  from  acquisition  community  to  the  warfighter.  In  order  to  be  most 
efficient  and  effective,  the  test  and  evaluation  process  must  complement  evolutionary  acqui¬ 
sition  and  spiral  development. 

A  technology  knowledge  base  is  essential.  The  acquisition  customers’  program  is 
best  served  if  a  repository  of  technology  development  efforts  were  available  to  improve  iden¬ 
tification  of  candidates  for  technology  insertion,  which  in  turn  may  accelerate  delivery  of  ma¬ 
ture  technology  to  our  Sailors  and  Marines. 

This  set  of  recommendations,  if  enacted  holistically,  will  result  in  Naval  research  en¬ 
terprise  that  is  held  in  high  esteem  and  confidence  by  the  acquisition  community — the  ulti¬ 
mate  consumer  of  S&T.  This  will  help  coalesce  S&T  and  acquisition  to  be  even  more  re¬ 
sponsive  to  the  warfighter  and  help  maintain  the  technology  advantage  the  US  enjoys  today. 
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Appendix  A 
Terms  of  Reference 
Technology  Acquisition  Reform 


Objective 

To  recommend  alternative  approaches  to  technology  acquisition  that  could  be  imple¬ 
mented  within  the  Department  of  the  Navy’s  acquisition  system. 

Background 

The  acquisition  system  used  by  the  Department  of  the  Navy  (DON)  to  procure  new 
systems  including  software  and  material  is  mandated  by  the  Department  of  Defense  (DoD). 
It  is  an  industrial  age  system  being  employed  in  an  information  age.  The  goal  of  the  acquisi¬ 
tion  system  is  to  insure  that  DON  personnel  have  the  best  and  most  reliable  sate  of  the  prac¬ 
tice  hardware  and  software  available  to  accomplish  assigned  missions.  Because  of  the  com¬ 
plexity  of  modem  day  warfare,  rate  of  change  of  technology,  bottom  line  emphasis  by  com¬ 
mercial  business,  as  well  as  regulatory  restrictions  both  internal  and  external  to  the  DoD,  the 
acquisition  system  has  become  cumbersome  and  excessive. 

In  addition  to  being  cumbersome  the  cost  of  administration  of  the  acquisition  system 
is  of  major  concern  both  for  government  and  industry  alike.  As  a  result  of  the  cost  and  com¬ 
plexity  of  maintaining  a  unique  DoD  acquisition  system,  there  are  commercial  companies 
who  will  not  seek  defense  contracts.  This  has  the  potential  of  denying  the  latest  technology 
to  operational  forces. 

Finally,  the  cost  of  administration  has  to  be  measured  in  the  currency  of  both  money 
and  time.  The  Milestone  Decision  Authorities  must  spend  considerable  time  reviewing  the 
necessary  infonnation  and  documentation  to  insure  that  the  acquisition  decisions  are  compli¬ 
ant  to  regulations.  In  many  cases  the  reviews  are  quickly  rendered  useless  by  changes  in  a 
program,  which  requires  new  documentation. 

Finally,  the  Chief  of  Naval  Operations  has  promulgated  the  strategy  of  Seapower  21. 
One  of  tenets  of  Seapower  21  is  to  speed  development  of  new  concepts  and  technologies.  In 
order  for  those  new  concepts  and  technologies  to  quickly  reach  the  intended  operating  forces 
there  must  be  streamlined  procedures  that  facilitate  rapid  implementation. 

Specific  Tasking 

This  NRAC  study  will  examine  current  approaches  to  managing  DON  acquisition 
programs  with  a  particular  emphasis  on  technology  acquisition.  This  study  will  also  examine 
alternative  approaches  tested  by  other  departments,  agencies,  and  countries.  Specifically, 
this  NRAC  study  will: 

•  Review  examples  of  new,  emerging,  and  experimental  technology  acquisition. 

For  example,  NMCI  lessons  learned,  CTTO  lessons  learned,  In-Q-Tel,  Army  venture 

fund,  DoD  and  Navy  venture  fund  plans,  UK  R&D  privatization. 
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•  Investigate  acquisition  alternatives  studied  by  ATL,  and  others. 

•  Recommend  procedures  and  opportunities  to  streamline  and  improve  technol¬ 
ogy  acquisition  subject  to  regulatory  restrictions. 
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Appendix  B 

Technology  Readiness  Levels 


Technology  Readiness  Levels,  long  in  use  at  the  National  Aeronautics  and  Space 
Administration  (NASA),  have  recently  been  adopted  across  the  Department  of  Defense. 
NASA  describes  them  as  “a  systematic  metric/measurement  system  that  supports  assess¬ 
ments  of  the  maturity  of  a  particular  technology  and  the  consistent  comparison  of  maturity 
between  different  types  of  technology.”  NASA  Management  Instruction  (NMI  7100)  for¬ 
mally  incorporates  TRLs  into  integrated  technology  planning  (John  C.  Mankins,  Advanced 
Concepts  Office,  Office  of  Space  Access  and  Technology,  NASA,  6  April  1995). 

TRL  1:  Basic  principles  observed  and  reported.  Lowest  level  of  technology  readi¬ 
ness.  Scientific  research  begins  to  be  translated  into  applied  research  and  development.  Ex¬ 
amples  might  include  paper  studies  of  a  technology’s  basic  properties. 

TRL  2:  Technology  concept  and/or  application  formulated.  Invention  begins.  Once 
basic  principles  are  observed,  practical  applications  can  be  invented.  The  application  is 
speculative  and  there  is  no  proof  or  detailed  analysis  to  support  the  assumption.  Examples 
are  still  limited  to  paper  studies. 

TRL  3:  Analytical  and  experimental  critical  function  and/or  characteristic  proof  of 
concept.  Active  research  and  development  is  initiated.  This  includes  analytical  studies  and 
laboratory  studies  to  physically  validate  analytical  predictions  of  separate  elements  of  the 
technology.  Examples  include  components  that  are  not  yet  integrated  or  representative. 

TRL  4:  Component  and/or  breadboard  validation  in  laboratory  environment.  Basic 
technological  components  are  integrated  to  establish  that  the  pieces  will  work  together.  This 
is  relatively  “low  fidelity”  compared  to  the  eventual  system.  Examples  include  integration  of 
“ad  hoc”  hardware  in  a  laboratory. 

TRL  5:  Component  and/or  breadboard  validation  in  relevant  environment.  Fidelity  of 
breadboard  technology  increases  significantly.  The  basic  technological  components  are  inte¬ 
grated  with  realistically  realistic  supporting  elements  so  that  the  technology  can  be  tested  in  a 
simulated  environment.  Examples  include  “high  fidelity”  laboratory  integration  of  compo¬ 
nents. 


TRL  6:  System/subsystem  model  or  prototype  demonstration  in  a  relevant  environ¬ 
ment.  Representative  model  or  prototype  system,  which  is  well  beyond  the  breadboard  tested 
for  TRL  5,  is  tested  in  a  relevant  environment.  Represents  a  major  step  in  a  technology’s 
demonstrated  readiness.  Examples  include  testing  a  prototype  in  a  high  fidelity  laboratory 
environment  or  in  simulated  operational  environment. 

TRL  7:  System  prototype  demonstration  in  an  operational  environment.  Prototype 
near  or  at  planned  operational  system.  Represents  a  major  step  up  from  TRL  6,  requiring  the 
demonstration  of  an  actual  system  prototype  in  an  operational  environment,  such  as  in  an  air¬ 
craft,  vehicle,  or  space.  Examples  include  testing  the  prototype  in  a  test  bed  aircraft. 
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TRL  8:  Actual  system  completed  and  “flight  qualified”  through  test  and  evaluation. 
Technology  has  been  proven  to  work  in  its  final  form  and  under  expected  conditions.  In  al¬ 
most  all  cases,  this  TRL  represents  the  end  of  true  system  development.  Examples  include 
developmental  test  and  evaluation  of  the  system  in  its  intended  weapon  system  to  determine 
if  it  meets  design  specifications. 

TRL  9:  Actual  system  “flight  proven”  through  successful  mission  operations.  Actual 
application  of  the  technology  in  its  final  fonn  and  under  mission  conditions,  such  as  those 
encountered  in  operational  test  and  evaluation.  In  almost  all  cases,  this  is  the  end  of  the  last 
“bug  fixing”  aspects  of  true  system  development.  Examples  include  using  the  system  under 
operational  mission  conditions. 
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Appendix  C 

Research,  Development  and  Evaluation  Budget  Activities 

Budget  activities  1,  2,  and  3  comprise  Science  and  Technology.  Budget  activities  4, 
5,  6,  and  7  form  that  portion  of  Research  Development  and  Evaluation  that  belong  to  the  Ac¬ 
quisition  system.  These  seven  budget  categories  are  commonly  referred  to  as,  respectively, 
“6.1,  6.2,  6.3,  6.4,  6.5,  6.6,  and  6.7.” 

The  following  definitions  of  Research  Development  and  Evaluation  Budget  Activities 
are  taken  from  the  Department  of  Defense  Financial  Management  Regulation  (DoD  7000.14- 
R)  Volume  2B,  Budget  Fonnulation  and  presentation,  Chapter  5,  Research,  Development  and 
Evaluation  Appropriations,  24  June  2002. 

Budget  Activity  1,  Basic  Research.  Basic  research  is  systematic  study  directed  toward 
greater  knowledge  or  understanding  of  the  fundamental  aspects  of  phenomena  and  of  observ¬ 
able  facts  without  specific  applications  towards  processes  or  products  in  mind.  It  includes  all 
scientific  study  and  experimentation  directed  toward  increasing  fundamental  knowledge  and 
understanding  in  those  fields  of  the  physical,  engineering,  environmental,  and  life  sciences 
related  to  long-tenn  national  security  needs.  It  is  farsighted  high  payoff  research  that  pro¬ 
vides  the  basis  for  technological  progress.  Basic  research  may  lead  to:  (a)  subsequent  applied 
research  and  advanced  technology  developments  in  Defense-related  technologies,  and  (b) 
new  and  improved  military  functional  capabilities  in  areas  such  as  communications,  detec¬ 
tion,  tracking,  surveillance,  propulsion,  mobility,  guidance  and  control,  navigation,  energy 
conversion,  materials  and  structures,  and  personnel  support.  Program  elements  in  this  cate¬ 
gory  involve  pre-Milestone  A  efforts. 

Budget  Activity  2,  Applied  Research.  Applied  research  is  systematic  study  to  under¬ 
stand  the  means  to  meet  a  recognized  and  specific  national  security  requirement.  It  is  a  sys¬ 
tematic  application  of  knowledge  to  develop  useful  materials,  devices,  and  systems  or  meth¬ 
ods.  It  may  include  design,  development,  and  improvement  of  prototypes  and  new  processes 
to  meet  general  mission  area  requirements.  Applied  research  translates  promising  basic  re¬ 
search  into  solutions  for  broadly  defined  military  needs,  short  of  system  development.  This 
type  of  effort  may  vary  from  systematic  mission-directed  research  beyond  that  in  Budget  Ac¬ 
tivity  1  to  sophisticated  breadboard  hardware,  study,  programming  and  planning  efforts  that 
establish  the  initial  feasibility  and  practicality  of  proposed  solutions  to  technological  chal¬ 
lenges.  It  includes  studies,  investigations,  and  non-system  specific  technology  efforts.  The 
dominant  characteristic  is  that  applied  research  is  directed  toward  general  military  needs  with 
a  view  toward  developing  and  evaluating  the  feasibility  and  practicality  of  proposed  solutions 
and  determining  their  parameters.  Applied  Research  precedes  system  specific  research.  Pro¬ 
gram  control  of  the  Applied  Research  program  element  is  normally  exercised  by  general 
level  of  effort.  Program  elements  in  this  category  involve  pre-Milestone  B  efforts,  also 
known  as  Concept  and  Technology  Development  phase  tasks,  such  as  concept  exploration 
efforts  and  paper  studies  of  alternative  concepts  for  meeting  a  mission  need. 
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Budget  Activity  3,  Advanced  Technology  Development  (ATD).  This  budget  activity 
includes  development  of  subsystems  and  components  and  efforts  to  integrate  subsystems  and 
components  into  system  prototypes  for  field  experiments  and/or  tests  in  a  simulated  envi¬ 
ronment.  ATD  includes  concept  and  technology  demonstrations  of  components  and  subsys¬ 
tems  or  system  models.  The  models  may  be  form,  fit  and  function  prototypes  or  scaled  mod¬ 
els  that  serve  the  same  demonstration  purpose.  The  results  of  this  type  of  effort  are  proof  of 
technological  feasibility  and  assessment  of  subsystem  and  component  operability  and  pro- 
ducibility  rather  than  the  development  of  hardware  for  service  use.  Projects  in  this  category 
have  a  direct  relevance  to  identified  military  needs.  Advanced  Technology  Development 
demonstrates  the  general  military  utility  or  cost  reduction  potential  of  technology  when  ap¬ 
plied  to  different  types  of  military  equipment  or  techniques.  Program  elements  in  this  cate¬ 
gory  involve  pre-Milestone  B  efforts,  such  as  system  concept  demonstration,  joint  and  Ser¬ 
vice-specific  experiments  or  Technology  Demonstrations.  Projects  in  this  category  do  not 
necessarily  lead  to  subsequent  development  or  procurement  phases. 

Budget  Activity  4,  Advanced  Component  Development  and  Prototypes  (ACD&P). 
Efforts  necessary  to  evaluate  integrated  technologies,  representative  modes  or  prototype  sys¬ 
tems  in  a  high  fidelity  and  realistic  operating  environment  are  funded  in  this  budget  activity. 
The  ACD&P  phase  includes  system  specific  efforts  that  help  expedite  technology  transition 
from  the  laboratory  to  operational  use.  Emphasis  is  on  proving  component  and  subsystem 
maturity  prior  to  integration  in  major  and  complex  systems  and  may  involve  risk  reduction 
initiatives.  Program  elements  in  this  category  involve  efforts  prior  to  Milestone  B  and  are 
referred  to  as  advanced  component  development  activities  and  include  technology  demon¬ 
strations.  Completion  of  Technology  Readiness  Levels  6  and  7  should  be  achieved  for  major 
programs.  Program  control  is  exercised  at  the  program  and  project  level.  A  logical  progres¬ 
sion  of  program  phases  and  development  and/or  production  funding  must  be  evident  in  the 
FYDP. 


Budget  Activity  5,  System  Development  and  Demonstration  (SDD).  SDD  programs 
have  passed  Milestone  B  approval  and  are  conducting  engineering  and  manufacturing  devel¬ 
opment  tasks  aimed  at  meeting  validated  requirements  prior  to  full-rate  production.  This 
budget  activity  is  characterized  by  major  line  item  projects  and  program  control  is  exercised 
by  review  of  individual  programs  and  projects.  Prototype  performance  is  near  or  at  planned 
operational  system  levels.  Characteristics  of  this  budget  activity  involve  mature  system  de¬ 
velopment,  integration  and  demonstration  to  support  Milestone  C  decisions,  and  conducting 
live  fire  test  and  evaluation  (LFT&E)  and  initial  operational  test  and  evaluation  (IOT&E)  of 
production  representative  articles.  A  logical  progression  of  program  phases  and  development 
and  production  funding  must  be  evident  in  the  FYDP  consistent  with  the  Department’s  full 
funding  policy. 

Budget  Activity  6,  RDT&E  Management  Support.  This  budget  activity  includes  re¬ 
search,  development,  test  and  evaluation  efforts  and  funds  to  sustain  and/or  modernize  the 
installations  or  operations  required  for  general  research,  development,  test  and  evaluation. 
Test  ranges,  military  construction,  maintenance  support  of  laboratories,  operation  and  main¬ 
tenance  of  test  aircraft  and  ships,  and  studies  and  analyses  in  support  of  the  RDT&E  program 
are  funded  in  this  budget  activity.  Costs  of  laboratory  personnel,  either  in-house  or  contractor 
operated,  would  be  assigned  to  appropriate  projects  or  as  a  line  item  in  the  Basic  Research, 
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Applied  Research,  or  Advanced  Technology  Development  program  areas,  as  appropriate. 
Military  construction  costs  directly  related  to  major  development  programs  are  included. 

Budget  Activity  7,  Operational  System  Development.  This  budget  activity  includes 
development  efforts  to  upgrade  systems  that  have  been  fielded  or  have  received  approval  for 
full  rate  production  and  anticipate  production  funding  in  the  current  or  subsequent  fiscal  year. 
All  items  are  major  line  item  projects  that  appear  as  RDT&E  Costs  of  Weapon  System  Ele¬ 
ments  in  other  programs.  Program  control  is  exercised  by  review  of  individual  projects.  Pro¬ 
grams  in  this  category  involve  systems  that  have  received  Milestone  C  approval.  A  logical 
progression  of  program  phases  and  development  and  production  funding  must  be  evident  in 
the  FYDP,  consistent  with  the  Department’s  full  funding  policy. 
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Appendix  D 

Glossary 

Acquisition.  The  conceptualization,  initiation,  design,  development,  test,  contract¬ 
ing,  production,  deployment,  logistic  support,  modification,  and  disposal  of  weapons  and 
other  systems,  supplies,  or  services  (including  construction)  to  satisfy  DoD  needs,  intended 
for  use  in  or  in  support  of  military  missions. 

Acquisition  Category  (AC  AT).  AC  AT  I  programs  are  Major  Defense  Acquisition 
Programs  (MDAPs).  An  MDAP  is  defined  as  a  program  estimated  by  the  Under  Secretary  of 
Defense  (Acquisition,  Technology  and  Logistics)  (USD(AT&L)  )  to  require  eventual  expen¬ 
diture  for  research,  development,  test,  and  evaluation  of  more  than  $365  million  (fiscal  year 
(FY)  2000  constant  dollars)  or  procurement  of  more  than  $2.19  billion  (FY  2000  constant 
dollars),  or  those  designated  by  the  USD(AT&L)  to  be  AC  AT  I.  AC  AT  I  programs  have  two 
sub-categories: 

•  ACAT  ID  for  which  the  Milestone  Decision  Authority  (MDA)  is 
USD(AT&L).  The  'D'  refers  to  the  Defense  Acquisition  Board  (DAB),  which  advises 
the  USD(AT&L)  at  major  decision  points. 

•  ACAT  IC  for  which  the  MDA  is  the  DoD  Component  Head  or,  if  dele¬ 
gated,  the  DoD  Component  Acquisition  Executive  (CAE).  The  'C  refers  to  Compo¬ 
nent. 

The  USD(AT&L)  designates  programs  as  ACAT  ID  or  ACAT  IC. 

ACAT  IA  programs  are  Major  Automated  Information  Systems  (MAISs)  or  programs 
designated  by  the  Assistant  Secretary  of  Defense  for  Command,  Control,  Communications, 
and  Intelligence  (ASD(C3I))  to  be  ACAT  IA.  A  MAIS  is  an  Automated  Infonnation  System 
(AIS)  program  that  is  (1)  designated  by  the  ASD(C3I)  as  a  MAIS,  or  (2)  estimated  to  require 
program  costs  in  any  single  year  in  excess  of  $32  million  (FY  2000  constant  dollars),  total 
program  in  excess  of  $126  million  (FY  2000  constant  dollars),  or  total  life  cycle  costs  in  ex¬ 
cess  of  $378  million  (FY  2000  constant  dollars).  MAISs  do  not  include  highly  sensitive  clas¬ 
sified  programs  (as  determined  by  the  Secretary  of  Defense)  or  tactical  communication  sys¬ 
tems.)  For  the  purpose  of  determining  whether  an  AIS  is  an  MAIS,  the  following  shall  be  ag¬ 
gregated  and  considered  a  single  AIS:  (1)  the  separate  AISs  that  constitute  a  multi-element 
program;  (2)  the  separate  AISs  that  make  up  an  evolutionary  or  incrementally  developed 
program;  or  (3)  the  separate  AISs  that  make  up  a  multi-component  AIS  program.  ACAT  IA 
programs  have  two  sub-categories: 

•  ACAT  IAM  for  which  the  MDA  is  the  Chief  Information  Officer 
(CIO)  of  the  Department  of  Defense  (DoD),  the  ASD(C3I).  The  ’M’  (in  ACAT  IAM) 
refers  to  Major  Automated  Information  System  (MAIS). 

•  ACAT  IAC  for  which  the  DoD  CIO  has  delegated  milestone  decision 
authority  to  the  CAE  or  Component  CIO.  The  'C  (in  ACAT  IAC)  refers  to  Compo¬ 
nent. 

The  ASD(C3I)  designates  programs  as  ACAT  IAM  or  ACAT  IAC. 
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ACAT  II  programs  are  defined  as  those  acquisition  programs  that  do  not  meet  the  cri¬ 
teria  for  an  ACAT  I  program,  but  do  meet  the  criteria  for  a  major  system.  A  major  system  is 
defined  as  a  program  estimated  by  the  DoD  Component  Head  to  require  eventual  expenditure 
for  research,  development,  test,  and  evaluation  of  more  than  $140M  in  FY2000  constant  dol¬ 
lars,  or  for  procurement  of  more  than  $660M  in  FY2000  constant  dollars  or  those  designated 
by  the  DoD  Component  Head  to  be  ACAT  II.  The  MDA  is  the  DoD  CAE. 

ACAT  III  programs  are  defined  as  those  acquisition  programs  that  do  not  meet  the 
criteria  for  an  ACAT  I,  an  ACAT  IA,  or  an  ACAT  II.  The  MDA  is  designated  by  the  CAE 
and  shall  be  at  the  lowest  appropriate  level.  This  category  includes  less-than-major  AISs. 

ACAT  IV  (Navy  and  Marine  Corps  only)  ACAT  programs  in  the  Navy  and  Marine 
Corps  not  otherwise  designated  as  ACAT  I,  II  or  III  are  designated  ACAT  IV.  There  are  two 
categories  of  ACAT  IV  programs:  IVT  and  IVM.  ACAT  IVT  programs  require  operational 
test  and  evaluation  while  ACAT  IVM  programs  do  not. 

Acquisition  community.  Within  the  Department  of  the  Navy,  those  offices  respon¬ 
sible  for  developing  and  acquiring  systems.  The  acquisition  community  includes  Program 
Executive  Offices,  Direct  Reporting  Program  Offices,  and  Systems  Commands.  This  com¬ 
munity  is  responsible  for  Budget  Activities  4  through  7.  The  term  is  also  often  used  in  an 
extended  sense  to  include  contractors  working  for  those  Naval  offices. 

Acquisition  Decision  Memorandum  (ADM).  A  memorandum  signed  by  the  mile¬ 
stone  decision  authority  (MDA)  that  documents  decisions  made  as  the  result  of  a  milestone 
decision  review,  decision  review,  or  interim  progress  review. 

Direct  Reporting  Program  Manager.  A  Program  Manager  who  reports  directly  to 
the  Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition)  as  opposed  to 
a  Program  Executive  Office. 

Disruptive  technology.  An  innovation  that  leads  to  new  classes  of  products  that  are 
cheaper,  better,  and  more  convenient  than  their  predecessors.  They  are  typically  adopted, 
initially,  only  by  a  small,  fringe  set  of  users,  and  rarely  emerge  in  response  to  customer  de¬ 
mand.  The  tenn  originated  with  Clayton  Christensen,  Professor  of  Business  Administration 
at  Harvard  Business  School  Disruptive  technologies  have  features  that  a  few  fringe  (and 
generally  new)  customers  value.  Products  based  on  disruptive  technologies  are  typically 
cheaper  to  produce,  simpler,  smaller,  better  perfonning,  and,  frequently,  more  convenient  to 
use. 

Evolutionary  acquisition.  Designated  as  the  preferred  (but  not  only)  acquisition  ap¬ 
proach  by  DoDI  5000.2.  Evolutionary  acquisition  is  an  acquisition  strategy  that  defines,  de¬ 
velops,  produces  or  acquires,  and  fields  an  initial  hardware  of  software  increment  (or  block) 
of  operational  capability.  It  is  based  on  technologies  demonstrated  in  relevant  environments, 
time-phased  requirements,  and  demonstrated  manufacturing  or  software  deployment  capa¬ 
bilities. 

Future  Naval  Capability  (FNC).  A  science  and  technology  program  designed  to  de¬ 
liver  capabilities  desired  by  the  Fleet  and  Force  over  the  next  seven  years.  Each  FNC  is  ap¬ 
proved  by  the  Department  of  the  Navy  Science  and  Technology  Corporate  Board  (the  Assis¬ 
tant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition),  the  Vice  Chief  of  Na¬ 
val  Operations,  and  the  Assistant  Commandant  of  the  Marine  Corps)  and  directed  by  an  Inte- 
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grated  Product  Team  (IPT)  whose  members  are  drawn  from  the  requirements,  acquisition, 
science  and  technology,  and  operational  communities.  The  IPT  identifies  enabling  capabili¬ 
ties  within  the  FNC,  and  then  gaps  within  those  enabling  capabilities.  The  science  and  tech¬ 
nology  community  representative  develops  and  manages  a  technical  program  to  fill  the  capa¬ 
bility  gaps  identified  by  the  IPT.  When  the  technologies  are  sufficiently  mature,  they  transi¬ 
tion  to  an  acquisition  program. 

Future  Years  Defense  Plan  (FYDP).  The  Future  Years  Defense  Plan  reflects  the 
decisions  associated  with  the  three  phases  of  the  PPBS. 

Integrated  Product  Team.  Team  composed  of  representatives  from  appropriate 
functional  disciplines  working  together  to  build  successful  programs,  identify  and  resolve 
issues,  and  make  sound  and  timely  recommendations  to  facilitate  decision  making.  There  are 
three  types  of  IPTs:  overarching  IPTs  (OIPTs)  that  focus  on  strategic  guidance,  program  as¬ 
sessment,  and  issue  resolution;  working  level  IPTs  (WIPTs)  that  identify  and  resolve  pro¬ 
gram  issues,  detennine  program  status,  and  seek  opportunities  for  acquisition  refonn;  and 
program  level  IPTs  that  focus  on  program  execution  and  may  include  representatives  from 
both  government  and  after  contract  award  industry. 

Major  Defense  Acquisition  Program  (MDAP).  An  acquisition  program  that  is  not 
a  highly  sensitive  classified  program  (as  determined  by  the  Secretary  of  Defense)  and  that  is 
designated  by  the  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 
(USD(AT&L))  as  an  MDAP,  or  estimated  by  the  USD(AT&L)  to  require  an  eventual  total 
expenditure  for  research,  development,  test  and  evaluation  (RDT&E)  of  more  than  365  mil¬ 
lion  in  fiscal  year  (FY)2000  constant  dollars  or,  for  procurement,  of  more  than  2.19  billion  in 
FY2000  constant  dollars. 

Milestone.  The  point  at  which  a  recommendation  is  made  and  approval  sought  re¬ 
garding  starting  or  continuing  an  acquisition  program,  i.e.,  proceeding  to  the  next  phase. 
Milestones  established  by  the  release  of  DoDI  5000.2  are:  MS  A,  that  approves  entry  into  the 
Concept  and  Technology  Development  phase;  MS  B,  that  approves  entry  into  the  System 
Development  and  Demonstration  phase;  and  MS  C,  that  approves  entry  into  the  Production 
and  Deployment  phase.  Also  of  note  is  the  Full  Rate  Production  Decision  Review  at  the  end 
of  the  Low  Rate  Initial  Production  work  effort  of  the  Production  and  Deployment  phase.  It 
authorizes  full  rate  production  and  approves  deployment  of  the  system  to  the  field  or  fleet. 

Milestone  Decision  Authority  (MDA).  The  individual  designated  in  accordance 
with  criteria  established  by  the  Under  Secretary  of  Defense  (Acquisition,  Technology  and 
Logistics)  (USD(AT&L)),  or  by  the  Assistant  Secretary  of  Defense  (Command,  Control, 
Communications,  and  Intelligence)  (ASD(C3I))  for  automated  infonnation  system  (AIS)  ac¬ 
quisition  programs,  to  approve  entry  of  an  acquisition  program  into  the  next  phase.  (DoDI 
5000.2) 

Planning,  Programming  and  Budgeting  System  (PPBS).  The  primary  resource  al¬ 
location  process  of  DoD.  One  of  three  major  decision  making  support  systems  for  defense 
acquisition.  It  is  a  formal,  systematic  structure  for  making  decisions  on  policy,  strategy,  and 
the  development  of  forces  and  capabilities  to  accomplish  anticipated  missions.  PPBS  is  a  cy¬ 
clic  process  containing  three  distinct,  but  interrelated  phases:  planning,  which  produces  De¬ 
fense  Planning  Guidance  (DPG);  programming,  which  produces  approved  program  objec- 
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tives  memorandum  (POM)  for  the  military  departments  and  defense  agencies;  and  budgeting, 
which  produces  the  DoD  portion  of  the  President's  Budget. 

Program  Executive  Officer  (PEO).  A  military  or  civilian  official  who  has  respon¬ 
sibility  for  directing  several  major  defense  acquisition  programs  and  for  assigned  major  sys¬ 
tem  and  non-major  system  acquisition  programs.  A  PEO  has  no  other  command  or  staff  re¬ 
sponsibilities  within  the  Component,  and  only  reports  to  and  receives  guidance  and  direction 
from  the  DoD  Component  Acquisition  Executive.  (DoDD  5000.1) 

Program  Manager  (PM).  The  individual  designated  in  accordance  with  criteria  es¬ 
tablished  by  the  appropriate  Component  Acquisition  Executive  to  manage  an  acquisition 
program,  and  appropriately  certified  under  the  provisions  of  the  Defense  Workforce  Im¬ 
provement  Act.  A  PM  has  no  other  command  or  staff  responsibilities  within  the  Component. 
(DoDD  5000.1). 

Program  Officer.  A  member  of  the  Office  of  Naval  Research  who  manages  science 
and  technology  programs  using  funds  from  Budget  Activities  1,  2,  or  3. 

Science  and  technology  community.  The  investors  and  performers  responsible  for 
Budget  Activities  1  through  3 :  basic  research,  applied  research,  and  advanced  technology  de¬ 
velopment.  The  Office  of  Naval  Research  leads  the  Naval  science  and  technology  commu¬ 
nity. 

Spiral  development.  An  iterative  process  for  developing  a  defined  set  of  capabilities 
within  one  increment.  This  process  provides  the  opportunity  for  interaction  between  the  user, 
tester,  and  developer.  In  this  process,  the  requirements  are  refined  through  experimentation 
and  risk  management,  there  is  continuous  feedback,  and  the  user  is  provided  the  best  possible 
capability  within  the  increment.  Each  increment  may  include  a  number  of  spirals.  Spiral  de¬ 
velopment  implements  evolutionary  acquisition. 

Sustaining  technology.  A  technology  that  improves  the  performance  of  established 
products.  Sustaining  technologies  are  usually  developed  by  successful  and  well  established 
companies  who  are  often  seen  as  holding  a  leadership  position  in  their  industries. 

Technology  insertion.  Introduction  of  a  newly  developed  or  newly  available  tech¬ 
nology  into  a  system  at  some  point  in  its  development  of  operational  life  cycle. 

Technology  transition.  Moving  technology  from  one  Budget  Activity  to  a  higher 

one. 

Technology  maturation.  Moving  technology  from  one  TRL  to  a  higher  one. 

Technology  Transition  Agreement.  An  agreement  concluded  between  a  science 
and  technology  program  and  an  acquisition  program  that  specifies  in  terms  of  TRL  and  de¬ 
tailed  cost,  schedule,  and  performance  metrics,  the  conditions  under  which  an  acquisition 
program  agrees  to  accept  a  technology  developed  in  a  science  and  technology  program. 

Technology  transfer.  Moving  technology  among  government  and  industrial  users  or 
producers. 

Transformational  technology.  A  technology  that  introduces  a  radically  new  capa¬ 
bility  that  renders  classes  of  existing  platforms,  systems,  or  conops  obsolete. 
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Appendix  E 

References 

The  National  Defense  University’s  research  librarians  compiled  the  following  sum¬ 
maries  of  reports  done  since  1997  that  bear  on  the  topic  of  technology  acquisition. 

Center  for  Naval  Analyses  Reports 

Getting  the  Most  Out  of  Integrated  Product  Teams  (IPTs)  (Center  for  Naval 
Analyses,  CRM-96-49- 1  0  October  1996).  The  report  recognizes  differences  between 
industry  and  government  IPTS  that  may  make  it  more  difficult  to  implement  EPTs  in 
the  Dept  of  Navy  including: 

Chain  of  command  considerations  that  hamper  free  and  open  discussion. 

•  Inability  to  fully  empower  government  comptrollers. 

It  recommends: 

•  Providing  guidance  and  boundaries.  Team  requirements  and  authori¬ 
ties  should  be  clear. 

•  Provide  initial  awareness/orientation  training  and  just-in-time 
team-building  and  special  skills  training. 

•  Using  full-time  and  collocated  IPTs  that  will  be  linked  to  organiza¬ 
tional  change  and  institutionalized  rather  than  operating  in  "meetings  only"  mode. 

•  The  broad  plan  for  IPTs  should  avoid  diluting  functional  expertise  by 
considering:  a.  The  impact  on  functional  organizations. 

•  The  need  to  retain  core  capabilities. 

•  The  need  to  continually  develop  the  workforce  and  retain  critical 

skills. 

•  The  need  to  maintain  a  "corporate  memory"  of  lessons  learned. 

•  Continually  reinforce  the  need  for  teaming  and  teamwork  by  recogniz¬ 
ing  rewarding  and  publicizing  good  examples  and  good  results. 

•  Restructure  IPTs  if  their  size  limits  their  effectiveness  by:  breaking 
large  IPTs  into  smaller  IPTs,  limiting  attendance  to  one  representative  from  each 
functional  organization  represented  on  the  team,  using  one  representative  empowered 
to  represent  more  than  one  functional  organization,  reassessing  the  need  for  represen¬ 
tatives  from  oraanizations  whose  possible  contributions  during  some  program  phases 
may  be  minimal. 

The  Revolution  in  Business  Affairs:  Realizing  the  Potential  (Center  for  Naval  Analy¬ 
sis  Corporation  1998  Annual  Conference,  Conference  Summary).  This  conference  featured 
several  prominent  speakers  such  as  John  White,  Jacques  Gansler,  John  Harare,  and  Jerry  Hul- 
tin.  The  Revolution  in  Business  Affairs  (RBA)  includes  many  initiatives  such  as  competitive 
sourcing,  acquisition  reform,  electronic  markets  and  activity-based  costing.  The  goal  is  to 
streamline  business  practices  within  the  Department  of  Defense.  The  conference  explored 
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such  topics  as  defining  the  RBA,  assessing  its  usefulness,  government  culture,  and  imple¬ 
menting  the  RBA.  There  are  no  overall  recommendations,  but  the  following  provides  a  brief 
summary  of  the  conference  proceedings.  There  were  individual  speakers,  and  several  panel 
discussions.  In  one  session,  six  military  leaders  discussed  their  views  of  the  Revolution  in 
Business  Affairs.  They  agreed  on  four  actions  that  senior  leaders  could  take  to  ensure  the 
success  of  the  RBA.  These  were: 

■  Defining  goals. 

■  Setting  the  organizational  climate. 

■  Ensuring  that  RBA  initiatives  provide  the  right  incentives. 

■  Once  the  plan  is  in  place,  get  out  of  the  way. 

Several  officials  from  the  Reagan  and  Bush  administrations  spoke  on  the  successes 
and  failures  of  previous  reform  efforts.  Some  of  the  "lessons  learned"  are  listed  below. 

■  Inclusion  of  DoD  top  management  is  critical. 

■  Expect  internal  resistance. 

■  The  cooperation  and  support  of  Congress  are  important. 

■  Progress  will  be  slow. 

Several  participants  addressed  the  role  of  competition  and  outsourcing  in  the  RBA. 
Much  of  the  discussion  concerned  the  A-76  process.  Acquisition  reform  was  another  major 
topic.  Stan  Soloway  (the  Deputy  Under  Secretary  of  Defense  for  Acquisition  Reform)  dis¬ 
cussed  the  three  elements  of  successful  acquisition  reform:  focus  more  on  improved  perform¬ 
ance,  effective  communication,  and  flexibility.  Other  speakers  discussed  individual  program 
efforts  (such  as  the  Apache  helicopter)  and  agency  efforts  (DLA,  Navy,  Air  Force). 

The  discussion  on  cost  control  centered  on:  Visibility  and  Management  of  Operating 
and  Support  Costs  (VAMOSC)  Activity-Based  Costing/Management  (ABC/MO  Earned 
Value  Management  (EVM). 

Cheryl  Kandaras  Chapman  led  a  discussion  on  improving  Congressional  support.  She 
suggested  four  questions  to  consider  for  discussion:  What  are  the  perceptions  of  barriers  to 
cooperation?  What  are  the  realities  of  barriers  to  cooperation?  What  are  our  definitions  of 
success?  What  is  the  effect  on  uniformed  and  civilian  personnel? 

In  the  final  session,  conference  participants  considered  a  future  scenario  where  none 
of  the  reform  efforts  were  ever  put  into  place.  The  question  is  "How  did  this  happen,  and  how 
can  we  prevent  it  from  happening?"  Some  of  the  speakers  believed  that  there  was  not  enough 
of  a  crisis  situation  to  motivate  change.  Others  cited  budget  problems  or  Congress  as  a  hin¬ 
drance  to  reform.  All  agreed  on  strong  leadership  to  gain  Congressional  support  and  begin 
the  reform  process. 
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Congressional  Research  Service  Summaries 

Navy-Marine  Corps  Tactical  Air  Integration  Plan:  Background  and  Issues  for 
Congress  (Christopher  Bolkcom  and  Ronald  O’Rouke.  CRS  RS21488,  10  April 
2003).  This  report  gives  background  and  issues  for  Congress  on  the  implementation 
of  a  Navy-Marine  Corps  Tactical  Air  Integration  (TAI)  plan  that  would  manage  Navy 
and  Marine  Corps  strike  fighters  as  a  common  pool  of  strike  fighters.  Issues  for  Con¬ 
gress: 

•  Total  Department  of  the  Navy  (DoN)  strike  fighter  capability  is  ques¬ 
tioned. 


—  Will  numerically  smaller  but  improved  TAI  force  be  able  to 
fight  and  win  two  overlapping  regional  conflicts? 

—  Does  “enhanced  funding”  mean  increased  funding  or  increased 
likelihood  of  funding?  If  increase  in  funding  is  less  than  DoN  believes, 
will  TAI  force  provide  the  same  capability? 

—  Will  improvements  to  aircraft  capability  really  make  it  more 
effective  than  the  previously  planned  force?  How  do  you  quantify  this 
improvement? 

—  The  TAI  plan  would  require  cross-training  of  Navy  and  Marine 
Corps  pilots.  What  effect  would  it  have  on  pilot  training  loads  or  the  abil¬ 
ity  of  pilots  to  achieve  high  levels  of  proficiency  in  specific  mission  areas? 

•  Cost  effectiveness  may  be  lower  due  to: 

—  Reductions  in  strike  fighter  procurement  ($1B)  may  be  offset 
by  increases  in  aircraft  operation  and  maintenance  costs  ($3.7B). 

—  Increased  spending  for  modernization  and  ancillary  equipment. 

—  Increased  unit  Joint  Strike  Fighter  (JSF)  procurement  costs  be¬ 
cause  of  lower  production  economies. 

•  Implications  for  future  aviation  integration: 

—  The  Marine  Corps  by  law  is  to  be  a  combined-arms  force  that 
includes  its  own  aviation  assets.  TAI  plans  appear  a  step  towards  turning 
Marine  fixed-wing  aircraft  over  to  the  Navy. 

—  TAI  plan  may  renew  discussion  on  cost  effectiveness  of  main¬ 
taining  separate  aviation  components. 


Defense  Acquisition  Reform:  Status  and  Current  Issues  (Valerie  Bailey 
Grasso,  CRS  IB96022,  8  November  2001).  The  post-Cold  War  period  necessitated 
defense  acquisition  reform,  and  this  report  presents  a  brief  history  and  general  over¬ 
view  of  those  measures.  In  general,  the  goals  are  to  the  make  the  Department  of  De¬ 
fense  (DoD)  acquisition  system: 

•  More  cost  effective. 
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•  Interactive  with  commercial  industries. 

•  Committed  to  procuring  state-of-the-art  technology  on  a  timely 

basis. 

The  report  then  lists  major  acquisition  issues  for  the  104th,  105th,  and  106th 
Congresses.  The  three  focus  areas  for  the  104th  Congress  were: 

•  Restructuring  DoD’s  acquisition  organizations  and  workforce. 

•  Outsourcing  acquisition-related  functions  to  the  private  sector. 

•  Overseeing  the  progress  of  the  Federal  Acquisition  Streamlin¬ 
ing  Act,  the  Federal  Acquisition  Reform  Act,  and  other  initiatives. 

Major  issues  for  the  105th  Congress  included: 

•  Reviewing  Congressionally  mandated  reports. 

•  Integrating  the  assessments  of  various  defense  panels. 

•  Assisting  DoD  in  implementing  acquisition  workforce  reduc¬ 
tions. 

The  106th  Congress  had  an  important  oversight  role  in  defense  acquisition  re¬ 
form,  especially  in  monitoring  DoD  reports  to  Congress. 

This  report  does  not  make  specific  recommendations. 

Defense  Science  Board  Reports 

Report  of  the  Defense  Science  Board  Task  Force  on  Acquisition  Reform  Phase 
IV  (July  1999).  This  paper  provides  metrics,  structured  in  four  tiers,  to  help  assess  de¬ 
partmental  progress  in  acquisition  refonn. 

•  Tier  I:  Establish  a  superior  military  capability  matched  to  the 
security  objectives  of  the  nation.  Acquire  superior  forces  at  a  reasonable  cost. 
A  future  defense  strategy  guides  resource  allocation  and  management  deci¬ 
sions.  Measure  our  forces  against  threat  scenarios  with  simulations,  war 
games,  and  actual  conflict  results.  Measure  annual  cost  to  sustain  United 
States  military  objectives  (track  DoD  budget)  to  ensure  reasonable  cost. 

•  Tier  II:  Choosing  the  “right”  things  to  acquire.  Increase  the 
mission  CINC’s  role  in  resource  allocation.  Use  the  user-supplier  resource 
balance  matrix  to  track  programs  and  resources. 

•  Tier  III:  Acquire  things  of  value  with  speed,  efficiency,  effec¬ 
tiveness  and  reduced  total  ownership  costs.  For  speed — measure  time  from 
program  initiation  to  initial  operational  capability  (IOC)  and  cycle  time  for 
system  upgrades  and  modifications.  For  effectiveness  and  efficiency — 
measure  the  percentage  of  programs  and  contracts  using  perfonnance  specifi¬ 
cations  without  detailed  statements  or  specification  and  contracts  using  price- 
based  acquisition  and  measure  amount  of  military  and  commercial  production 
integration  occurring  by  sector.  For  reduced  ownership  costs — measure  five 
programs  that  make  total  ownership  costs  or  life  cycle  cost  a  requirement,  sys- 
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terns  that  effectively  track  total  ownership  costs  and  benchmarks  to  best  in 
class. 


•  Tier  IV:  Maintain  the  public  trust.  Broaden  the  use  of  competi¬ 
tion  tracking  dollar  value  of  investment  programs  and  fielded  systems.  Estab¬ 
lish  the  value  of  things  to  be  acquired  through  user  involvement.  Depend  on 
competitive  market  forces.  Assess  public  perception  of  the  acquisition  proc¬ 
ess. 

Report  of  the  Defense  Science  Board  Task  Force  on  Vertical  Integration  and 
Supplier  Decisions  (May  1997).  This  report  recommends: 

•  Expand  the  Department’s  monitoring  of  vertical  supply  relationships 
for  selected  important  defense  products  and  technologies. 

•  Focus  DoD  acquisition  and  technology  strategies  and  investments  to 
support  competition  and  innovation. 

•  During  antitrust  reviews,  continue  to  scrutinize  carefully  the  potential 
hanns  from  vertical  integration. 

•  Strengthen  business-  and  industry-related  skills  of  the  Department’s 
acquisition  personnel. 

•  Develop  measures  that  help  DoD  managers  to  recognize  areas  of  po¬ 
tential  vertical  integration  concern  and  trigger  more  detailed  investigation. 

Appendix  D  to  the  Report  of  the  Defense  Science  Board  on  Defense  Acquisi¬ 
tion  Reform  (Phase  II)  Jet  Engine  Commercial  Practices  Panel  Final  Report  (13  May 
1994).  This  report  recommends: 

•  A  detailed  comprehensive  program  should  be  established  to  convert 
the  military  jet  engine  industry  to  commercial  practices  for  procuring  and  support¬ 
ing  mature  engine,  production,  and  support  programs. 

•  The  Administration,  Congress,  and  the  Department  of  Defense  should 
provide  the  necessary  waivers  and  exceptions  to  the  various  laws,  regulations, 
standards,  and  specifications  that  will  allow  pure  commercial  practices  to  be  used 
to  procure  and  support  mature  military  engine  production  and  support  programs. 

•  A  joint  government  and  industry  team,  under  the  direction  of  the  Dep¬ 
uty  Under  Secretary  of  Defense  (Acquisition  Reform)  should  be  established  and 
funded  to  implement  the  program.  The  team  would  create  a  detailed,  time-phased 
plan  for  commercial  practices  on  current  programs  (where  practical),  follow-on 
procurements  of  current  in-production  engines,  and  on  future  engines  as  they 
complete  qualification  and  enter  production.  The  team  would  also  explore  oppor¬ 
tunities  to  implement  commercial  practices  during  jet  engine  development. 

Report  of  the  Defense  Science  Board  Task  Force  on  Defense  Acquisition  Re¬ 
form  (Phase  II)  (August  1994).  The  report  recommends  pilot  industry  initiatives: 

•  Develop  a  funded  government-industry  program  to  transition  current 
contracts  and  programs  to  a  commercial  practice  basis. 
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•  Extend  to  the  whole  of  DoD  the  recent  internal  Air  Force  study  on 
streamlined  acquisition  of  microelectronics. 

•  Use  the  software  acquisition  policy  recommended  by  the  Defense  Sci¬ 
ence  Board  (DSB)  Task  Force  on  Software  as  the  basis  for  commercialization  in 
this  industrial  sector. 

•  Bring  the  National  Reconnaissance  Office  (NRO)  -  Industry  Panel 
studying  the  use  of  commercial  practices  in  the  DoD  space  industry  to  the  point  of 
actionable  recommendations. 

Report  of  the  Defense  Science  Board  Task  Force  on  Acquisition  Reform  Phase 
IV  Subpanel  on  Research  and  Development  (July  1999).  After  assessing  Acquisition 
Reform  Initiatives,  the  Task  Force  recommends  that  implementation  of  the  Price- 
Based  Acquisition  Model  as  the  principal  method  for  development  and  procurement 
of  DoD  systems.  The  Task  Force  also  examined  specific  programs  and  made  addi¬ 
tional  recommendations  as  follows: 

•  Joint  Strike  Fighter — insure  continuous  competition  throughout  the 
procurement  and  sustainment  cycle. 

•  Shipbuilding — the  DD  2 1  should  be  the  designated  model  for  acquisi¬ 
tion  refonn  practices. 

•  Space — use  commercially  available  capabilities  whenever  possible. 
DoD  research  efforts  should  focus  on  military-specific  technology  not  commer¬ 
cially  available. 

•  Joint  Tactical  Radio  Program — define  an  acceptable  technical  archi¬ 
tecture  based  on  commercial  specifications. 

•  Education  and  training — responsibility  for  acquisition  education  and 
training  should  be  clarified  and  strengthened. 

Defense  Science  and  Technology — 2001  Summer  Study  (2002).  The  report 
provides  an  overview  of  science  and  technology  resources  within  the  Department  of 
Defense.  The  goal  is  to  accomplish  transfonnation  of  military  capabilities,  and  transi¬ 
tion  by  rapid  insertion  of  technology.  “Operational  experimentation,  spiral  develop¬ 
ment,  and  rapid  evolutionary  acquisition”  should  be  used  to  transform  the  science  and 
technology  environment.  The  report  made  further  recommendations  as  follows: 

•  Maintain  current  level  of  science  and  technology  investment. 

•  New  science  and  technology  initiatives  to  meet  four  transformational 
challenges.  These  are  defense  against  biological  warfare,  finding  difficult  targets, 
decision-making,  and  enabling  high-risk  operations. 

•  Exploit  commercial  technology. 

•  Spiral  development  process — a  five-year  maximum  acquisition  time. 

•  Extensive  use  of  red  teams. 

•  Establish  a  new  technology  transition  process. 
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•  Accelerate  the  transition  process  for  joint  research  and  development 

•  Restructure  the  Department  of  Defense  laboratories. 

Defense  Acquisition  Reform,  Phase  III:  A  Streamlined  Approach  to  Weapons 
Systems  Development,  Research  and  Acquisition.  The  Application  of  Commercial 
Practices  (May  1996).  The  current  Department  of  Defense  system  is  outmoded,  and 
should  be  replaced  with  the  best  commercial  practices.  Following  the  Phase  I  and  II 
reports,  the  focus  of  Phase  III  is  on  extending  best-of-class  practices  to  the  research 
and  development  stages  of  acquisition.  The  Commercial-Style  Research  and  Devel¬ 
opment  Model  is  the  recommended  method  for  doing  this. 

•  Implement  commercial  practices  into  the  research  and  development 
stages  of  acquisition  (based  on  the  American  free-market  economy). 

•  Use  a  phased  approach  or  model  to  implement  best  practices  and  de¬ 
crease  time  of  acquisition  cycle  (Commercial-Style  Research  and  Development 
Model). 

•  Institutionalized  decision-making — the  Vice  Chair,  Joint  Chiefs  of 
Staff,  as  focal  point  for  users  (the  Services)  and  Under  Secretary  of  Defense  (Ac¬ 
quisition  and  Technology)  as  representative  for  suppliers.  They  should  work  to¬ 
gether  to  make  the  most  informed  decision. 

•  Measures  to  protect  Government  interests  and  maintain  the  public  trust 
include  continuous  competition,  flexible  perfonnance  contracts,  risk  reduction 
phase,  and  evaluation  of  past  contractor  performance. 

Report  of  the  Defense  Science  Board  Task  Force  on  Acquiring  Defense  Soft¬ 
ware  Commercially  (June  1994).  The  report  recommends: 

•  Application  of  commercial  practices  to  software  acquisition. 

•  A  “centralized  approach”  for  greater  management  control  and  over¬ 
sight,  with  the  Under  Secretary  of  Defense  (Acquisition  and  Technology)  respon¬ 
sible  for  software  policy  and  program  implementation. 

•  Formation  of  an  Executive  Council,  and  a  supporting  “process  action 
team”  to  implement  Task  Force  recommendations. 

•  Notable  on-going  efforts  include  the  Air  Force  Electronic  Systems 
Command,  the  Reserve  Component  Automation  System,  and  the  Global  Com¬ 
mand  and  Control  System. 

Report  of  the  Defense  Science  Board  Task  Force  on  Defense  Acquisition  Re¬ 
form,  Phase  I  (July  1993).  This  report  recommends  a  complete  overhaul  of  the  De¬ 
partment  of  Defense  acquisition  system,  to  increase  efficiency,  include  state-of-the- 
art  technology,  and  strengthen  the  United  States  industrial  base.  The  Task  Force  rec¬ 
ommends  the  application  of  commercial  practices  to  the  maximum  extent  possible,  in 
a  manner  that  continues  to  maintain  the  public  trust.  Recommendations  include: 

•  Move  away  from  the  Department  of  Defense  cost-based  acquisition 
system  (DFARS  211  relaxes  the  requirement  for  cost-based  acquisition). 
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•  Broaden  procurement  of  commercial  practices. 

•  Use  simplified  procurement  practices. 

•  Reduce  reliance  on  cost  or  pricing  data. 

•  Select  specific  industrial  sectors,  and  implement  commercial  practices 
in  those  sectors.  Bring  together  the  private  and  public  participants  within  these 
sectors. 


•  Select  two  major  Unified  Commands  and  increase  their  capabilities  for 
technology  insertion  and  requirements  definition. 

•  The  first  annual  report  detailing  goals,  responsibilities,  and  progress 
should  be  ready  by  January  1994. 

•  Establish  an  outside  Review  Group  to  provide  oversight. 

•  Establish  a  comprehensive  education  program  for  government,  indus¬ 
try,  and  the  public. 

General  Accounting  Office  Reports 

Defense  Acquisitions:  Need  to  Revise  Acquisition  Strategy  to  Reduce  Risk  for 
Joint  Air-to-Surf ace  Standoff  Missile  (Letter  Report,  26  April  2000,  GAO/NSIAD-OO- 
75).  Recommendations  include  linking  production  decisions  more  closely  to  knowl¬ 
edge  points.  The  report  also  recommends  moving  from  an  arbitrary  production  date 
to  ensure  that  before  beginning  production  the  missile  design  is  stable,  flight  testing 
establishes  the  missile’s  ability  to  meet  perfonnance  requirements,  and  key  manufac¬ 
turing  processes  are  controlled  so  quality,  cost,  and  volume  are  proven  and  accept¬ 
able. 

Defense  Acquisitions :  Decisions  on  the  Joint  Strike  Fighter  will  be  Critical  for 
Acquisition  Reform  (Testimony,  10  May  2000,  GAO/T-NSIAD-OO-173).  The  study 
recommends  continuing  the  Joint  Strike  Fighter  program  in  its  current  definition  and 
risk  reduction  phase,  and  delaying  the  decision  to  move  into  engineering  and  manu¬ 
facturing  until  technologies  are  demonstrated  to  acceptable  levels. 

Defense  Acquisition:  Improved  Program  Outcomes  are  Possible  (Testimony, 
18  March  1998,  GAO/T-NSIAD-98-123).  Decision-makers  must  provide  incentives 
to  drive  behaviors  by  accepting  collective  responsibility  for  incentives  that  drive  be¬ 
havior,  placing  a  cultural  focus  on  “why” — or  the  incentives  that  affect  behavior — to 
complement  the  “how”  (process  and  control)  and  the  “who”  (organization),  and  ac¬ 
cepting  that  actions  taken  and  decisions  made  on  individual  programs  communicate 
the  broader  message  of  “what  will  work”  to  others  in  the  process.  Congress  and  the 
Department  of  Defense  can  depressurize  program  launch  decisions  by  relieving  the 
need  to  overpromise  on  perfonnance  and  resource  estimates,  and  can  make  it  accept¬ 
able  for  program  managers,  once  a  program  is  underway,  to  identify  unknowns  as 
high  risks  so  they  can  be  worked  on  earlier  in  development.  The  Secretary  of  De¬ 
fense  can  redefine  the  point  for  launching  programs  as  the  point  at  which  technology 
development  ends  and  product  development  begins.  For  individual  program  deci¬ 
sions,  the  Secretary  should  send  the  signals  that  create  incentives  for  acquisition  man- 
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agers  to  identify  unknowns  and  ameliorate  their  risks  early  in  development.  Finally, 
the  Secretary  should  develop  a  policy  that  promotes  productive  supplier  relationships 
and  emphasizes  the  importance  of  suppliers  in  improving  acquisition  outcomes  and 
communicate  this  policy  through  acquisition  workforce  and  the  defense  industry. 
Specific  recommendations  to  Congress  include  supporting  the  Secretary  of  Defense’s 
efforts  to  change  the  environment  by  changes  to  the  acquisition  process  that  provide 
incentive  for  gaining  sufficient  knowledge  at  key  points  in  weapon  acquisition  pro¬ 
grams,  providing  funds  to  manage  technology  development  efforts  outside  the  bounds 
of  individual  programs  (if  the  Secretary  of  Defense  separates  technology  development 
from  product  development),  and  helping  create  the  right  incentives  for  individual 
programs  by  favorably  considering  the  Department  of  Defense  funding  requests  to 
mitigate  high  risks  early  in  a  program. 

Acquisition  Reform:  DoD ’s  Guidance  on  using  Section  845  Agreements  Could 
be  Improved  (Letter  Report,  7  April  2000,  GAO/NSIAD-OO-33).  The  report  recom¬ 
mends  that  the  Secretary  of  Defense  provide  updated  guidance  that  lays  out  condi¬ 
tions  for  using  Section  845  agreements  and  provide  a  framework  to  tailor  the  terms 
and  conditions  appropriate  for  each  agreement.  The  Secretary  should  also  establish 
and  require  the  use  of  a  set  of  metrics,  including  the  number  of  commercial  firms  par¬ 
ticipating  in  Section  845  agreements,  which  are  measurable  and  directly  related  to  the 
agreement.  Requirements  should  be  in  place  in  time  to  assist  in  deliberations  on 
whether  to  extend  the  authority  past  30  September  2001. 

Defense  Acquisition:  Best  Commercial  Practices  Can  Improve  Program  Out¬ 
comes  (Statement/Record,  17  March  1999,  GAO/T-NSIAD-99-1 16).  This  report  is 
primarily  an  update  on  Department  of  Defense  progress  toward  improving  the  way  it 
acquires  weapon  systems.  Efforts  at  systemic  change  have  not  been  reflected  in  man¬ 
agement  and  decision-making  on  individual  programs.  The  report  suggests  a  series  of 
actions  aimed  at  fostering  an  environment  in  the  Department  of  Defense  that  encour¬ 
ages  or  rewards  best  practices.  This  environment  must  become  conducive  to  adopting 
best  commercial  practices  for  gaining  knowledge  and  assessing  risks.  Program 
launch  decisions  must  be  relieved  of  pressure  to  overpromise  on  performance  and  re¬ 
source  estimates.  Once  a  program  is  underway,  participants  in  the  acquisition  process 
must  make  it  acceptable  for  managers  to  identify  unknowns  as  high  risks  so  that  they 
can  be  aggressively  worked  earlier.  Decisions  made  on  individual  weapon  systems 
send  a  strong  message  about  the  Department  of  Defense’s  definition  of  success  and 
practices  that  lead  to  success. 

Best  Practices:  Better  Acquisition  Outcomes  are  Possible  if  DoD  Can  Apply 
Lessons  from  F/A-22  Program  (11  April  2003,  GAO-03-645T).  This  testimony  does 
not  make  recommendations.  It  only  compares  best  practices  with  actual  experience 
of  the  F/A-22  program.  Department  of  Defense  emphasizes  the  use  of  evolutionary, 
knowledge-based  acquisitions  concepts  in  the  newest  acquisition  policy.  Most  De¬ 
partment  of  Defense  programs  currently  do  not  employ  these  practices  and,  as  a  re¬ 
sult,  experience  cost  increases,  schedule  delays,  and  poor  product  quality  and  reliabil¬ 
ity.  Leadership  commitment  and  attention  to  putting  the  policy  into  practice  for  indi¬ 
vidual  programs  is  needed  to  avoid  the  problems  of  the  past. 
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Defense  Reform  Initiative:  Organization,  Status,  and  Challenges  (Chapter  Re¬ 
port,  21  April  1999,  GAO/NSIAD-99-87).  The  General  Accounting  Office  assesses 
progress  in  the  Department  of  Defense  implementation  of  the  Defense  Reform  Initia¬ 
tive  (DRI),  a  program  to  establish  modem  business  practices  within  the  Defense  ac¬ 
quisition  community.  The  report  notes  that  the  DRI  has  only  been  in  place  a  short 
while,  and  that  some  progress  has  been  made.  The  recommendations  identify  areas 
for  improvement: 

•  Include  other  reform  efforts. 

•  Develop  an  integrated  strategy  and  action  plan  for  managing 

the  DRI. 

•  Identify  funding  requirements  and  target  areas  for  the  program. 

The  report  also  evaluates  the  progress  of  the  four  DRI  pillars.  The  first  pillar 
is  reengineering  defense  business  and  support  functions  by  implementing  private  sec¬ 
tor  best  practices.  This  includes  a  wide  range  of  activities,  such  as  the  use  of  paper¬ 
less  contracting,  electronic  commerce,  purchase  cards,  reengineering  the  DoD’s  travel 
system,  and  transportation  of  military  members’  household  goods.  While  progress 
has  been  made,  some  aspects  of  this  pillar  will  take  several  years  to  complete  and  will 
not  meet  the  DRI  program  milestones. 

The  second  pillar  is  reorganizing  and  reducing  the  size  of  DoD  headquarters 
and  agencies.  There  has  been  significant  progress.  Plans  are  being  developed  to  meet 
Congressionally  mandated  reduction  of  25%  of  headquarters  management. 

The  third  pillar,  expanding  the  use  of  competitive  sourcing,  has  a  focus  of 
shifting  thousands  of  government  positions  to  the  private  sector.  The  plan  includes 
examining  229,000  positions  for  possible  conversion,  which  will  result  in  cost  sav¬ 
ings  for  the  DoD. 

The  fourth  pillar  calls  for  reducing  Defense  infrastructure,  with  a  focus  on 
conducting  two  additional  Base  Realignment  and  Closure  (BRAC)  rounds.  The 
BRAC  rounds  have  not  been  approved  by  Congress,  but  other  reduction  methods  are 
being  used,  such  as  agency  consolidations,  demolition  of  excess  facilities,  and  priva¬ 
tization  of  utilities. 

Federal  Acquisition:  Trends,  Reforms,  and  Challenges  (Testimony,  16  March 
2000,  T-OGC-OO-7,  Statement  of  Henry  L.  Hinton,  Jr.,  Assistant  Comptroller  Gen¬ 
eral,  National  Security  and  International  Affairs  Division.  In  this  prepared  statement, 
Mr.  Hinton  discusses  the  changing  acquisition  environment,  recent  reform  efforts, 
and  explores  current  and  future  challenges  in  federal  acquisition.  In  the  current  envi¬ 
ronment,  DoD  remains  the  largest  purchaser,  accounting  for  about  two  thirds  of  all 
federal  contracting  dollars.  Overall,  agencies  spend  more  on  services  than  on  equip¬ 
ment  and  supplies.  The  acquisition  process  has  become  more  streamlined,  but  many 
problems  remain,  and  the  system  is  still  slow  and  inefficient.  Trends  include  down¬ 
sizing,  privatization,  and  e-commerce. 

The  recent  history  of  reform  efforts,  many  initiated  in  the  eighties,  shows  dif¬ 
ferent  programs  and  legislation  that  regulated  every  aspect  of  the  acquisition  process. 
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The  result  was  a  complex  and  unwieldy  system  little-suited  for  the  commercial  busi¬ 
ness  environment.  In  response,  Congress  enacted  the  Federal  Acquisition  Streamlin¬ 
ing  Act  of  1994  and  the  Clinger-Cohen  Act  of  1996.  The  full  effects  of  these  reforms 
cannot  yet  be  measured. 

There  are  three  major  challenges  facing  the  government:  improving  the  out¬ 
comes  of  Defense  systems  acquisitions,  acquiring  and  using  information  technology, 
and  addressing  acquisition  workforce  issues. 

Joint  Strike  Fighter  Acquisition:  Mature  Critical  Technologies  Needed  to  Re¬ 
duce  Risks  (GAO-02-39,  October  2001).  This  report  assesses  critical  technologies  for 
the  Joint  Strike  Fighter  program.  The  technologies  are  not  mature  (have  not  been 
built  to  size  and  tested  under  the  conditions  of  the  actual  product,  and  are  not  ready 
for  integration)  and  therefore  present  a  high  risk  to  the  Joint  Strike  Fighter  program  as 
it  moves  into  the  engineering  and  manufacturing  phase.  This  is  also  inconsistent  with 
best  practices.  The  report  recommends  delaying  the  engineering  and  manufacturing 
development  until  critical  technologies  are  mature. 

Best  Practices:  Taking  a  Strategic  Approach  Could  Improve  DoD  ’s  Acquisi¬ 
tion  of  Sendees  (18  January  2002,  GAO-02-230).  The  General  Accounting  Office 
studied  the  purchasing  practices  of  six  leading  United  States  companies,  including 
Exxon  Mobil,  Hasbro,  and  Merrill  Lynch.  All  of  the  companies  studied  had  reengi¬ 
neered  their  purchasing  practices  and  achieved  outstanding  results.  Noting  that  the 
Department  of  Defense  has  already  begun  to  incorporate  commercial  best  practices, 
and  recognizing  the  unique  differences  of  the  Department  of  Defense,  the  report  rec¬ 
ommends  that  the  Secretary  of  Defense  should  evaluate  a  strategic  reengineering  ap¬ 
proach  using  corporate  practices  as  a  framework  for  the  Defense  reengineering  effort. 
Specifically,  the  Secretary  should  assess  whether  current  or  planned  financial  man¬ 
agement  systems  can  produce  the  data  necessary  for  analysis,  and  whether  the  current 
organizational  structure  is  adequate  and  properly  used.  The  Secretary  should  also  as¬ 
sess  processes  and  roles  for  the  most  efficient  acquisition  of  services. 

Defense  Space  Activities:  Organizational  Changes  Initiated,  but  Further 
Management  Actions  Needed  (18  April  2003,  GAO-30-379).  The  Commission  to  As¬ 
sess  United  States  National  Security  Space  Management  and  Organization  (the  Space 
Commission)  made  recommendations  to  the  Department  of  Defense  in  2001.  This 
report  updates  and  earlier  (June  2002)  assessment  of  Defense  efforts  to  implement 
these  recommendations.  Several  of  the  original  thirteen  recommendations  have  been 
completed.  Important  organizational  changes  include: 

•  Appointing  the  Under  Secretary  of  the  Air  Force  also  as  Director,  Na¬ 
tional  Reconnaissance  Office. 

•  Realigning  Air  Force  space  activities  under  one  command. 

•  Creating  a  separate  position  of  Commander,  Air  Force  Space  Com¬ 
mand. 

Additional  recommendations  include: 

•  Develop  a  national  security  space  strategic  plan. 
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•  Establish  a  strategic  approach  for  space  human  capital. 

•  Designate  a  department-level  entity  to  provide  space  program 
oversight  and  assess  progress. 

Defense  Acquisitions:  Assessments  of  Major  Weapon  Programs  (15  May 
2003,  GAO-03 -476).  This  is  the  first  of  an  annual  report  providing  Congressional  and 
Defense  decision-makers  with  a  knowledge-based  assessment  of  Defense  programs 
that  identifies  risks  and  highlights  best  practices.  Twenty-six  major  weapons  pro¬ 
grams  are  studied.  There  are  no  recommendations,  but  program  office  comments  are 
included  in  each  individual  assessment. 

Defense  Acquisitions:  Factors  Affecting  Outcomes  of  Advanced  Concept 
Technology  Demonstrations  (2  December  2002,  GAO-03-52).  The  Advanced  Con¬ 
cept  Technology  Demonstration  (ACTD)  program  was  started  by  the  Department  of 
Defense  as  a  way  to  get  new  technologies  that  meet  critical  military  needs  into  the 
hands  of  users  faster  and  for  less  cost.  The  General  Accounting  Office  was  asked  to 
examine  the  Department  of  Defense’s  process  for  structuring  and  executing  ACTDs. 
Under  the  ACTD  program,  a  prototype  is  built  and  tested  under  realistic  conditions, 
and  then  is  approved  or  rejected  by  the  Department  of  Defense.  Use  of  the  ACTD 
can  reduce  the  time  of  development  from  ten  to  fifteen  years,  to  about  two  to  six 
years.  The  study  looked  at  twenty-four  projects,  and  made  the  following  recommen¬ 
dations: 


•  The  Department  of  Defense  should  strengthen  its  criteria  for  assessing 
the  military  utility  of  ACTD  projects. 

•  Consider  ways  to  ensure  funding  is  provided  for  acquisitions. 

•  The  Department  of  Defense  should  seek  input  from  the  Secretary  of 
Defense  on  whether  to  transition  tested  technologies. 

Information  Technology:  DoD  Needs  to  Leverage  Lessons  Learned  from  its 
Outsourcing  Projects  (25  April  2003,  GAO-03-371).  The  study  researched  leading 
commercial  practices  for  the  outsourcing  of  information  technology  services  and  pub¬ 
lished  a  framework  of  seven  phases  that  included  ah  of  the  activities  involved  in  out¬ 
sourcing.  The  study  then  examined  whether  Defense  outsourcing  projects  were  using 
best  commercial  practices,  and  whether  these  lessons  were  being  shared  across  the 
Department  of  Defense.  Recommendations  include: 

•  Establish  an  electronic  tool  to  “capture”  lessons  learned  from  success¬ 
ful  information  technology  outsourcing  projects  for  future  reference. 

•  Incorporated  the  main  elements  of  a  lessons-learned  program — 
collection,  verification,  storage,  and  dissemination. 

•  The  Under  Secretary  of  Defense  (Acquisition,  Technology,  and  Logis¬ 
tics)  and  the  Assistant  Secretary  of  Defense  (Command,  Control,  Communication, 
and  Intelligence)  should  work  together  on  this. 
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RAND  Reports 

Assessing  the  Use  of  '‘Other  Transactions”  Authority  for  Prototype  Projects  (Giles 
K.  Smith,  Jeffrey  Drezner,  and  Irving  Lachow,  RAND  DB-375-OSD,  2002).  This  study  ex¬ 
amined  21  projects  using  the  “other  transactions”  (OT)  process  and  found  it  provided  im¬ 
proved  access  to  important  commercial  technologies  as  well  as  improved  efficiencies  in  con¬ 
ducting  technological  risky  prototype  projects.  It  concluded: 

•  Segment  of  major  firms  that  were  formerly  focused  exclusively  on 
commercial  projects  are  now  willing  to  participate  in  DoD  prototype  projects  because 
of  the  freedoms  inherent  in  the  OT  process. 

•  Flexibility  of  the  OT  process  can: 

—  Achieve  better  use  of  industry  resources  through  innovative  busi¬ 
ness  arrangements  and  project  designs. 

—  Improve  management  of  risks  and  uncertainties  through  freedom  to 
modify  the  program  as  it  evolves. 

—  Achieve  better  value  through  cost-sharing  and  reduction  of  transac¬ 
tion  costs.  More  effort  is  being  devoted  to  product  and  less  to 
process. 

•  Some  risks  to  the  government  are  incurred  due  to  less  access  to  firms’ 
financial  records  and  ownership  of  intellectual  property  but  immediate  rewards  out¬ 
weigh  risks. 


Strategic  Sourcing:  Measuring  and  Managing  Performance  (Laura  H.  Bald¬ 
win,  Frank  A.  Camm,  and  Nancy  Y.  Moore,  Rand  DB-287-AF.  2000).  A  strategic 
sourcing  process  links  the  decisions  made  during  the  sourcing  process  to  the  strategic 
goals  of  the  customer  organization.  The  Air  Force  can  adapt  commercial  firms’  prac¬ 
tices  to  improve  effectiveness  of  services  acquisition. 

Use  metrics  to  focus  its  limited  sourcing  resources  on  activities  that  appear  to  offer  the  high¬ 
est  potential  returns  in  perfonnance  and  cost  improvements.  Audits  that  baseline  internal  per¬ 
formance  and  cost  and  benchmarking  studies  that  identify  external  capabilities  can  facilitate 
this  process. 

•  Use  metrics  to  focus  the  source  selection  decision  on  aspects  of  per¬ 
formance  and  cost  that  best  support  goals  of  the  Air  Force  customer  organization. 

•  Use  metrics  to  better  manage  its  relationships  with  external  providers. 
Air  Force  can  build  mutual  trust  with  providers  and  reduce  manpower  and  other  costs 
of  the  quality  assurance  program:  use  metrics  that  support  strategic  goals,  set  aggres¬ 
sive  goals  for  the  metrics,  and  manage  performance  through  open,  frequent  commu¬ 
nication  and  problem  solving  supplemented  by  longer-term  efforts  to  improve  per¬ 
formance 

•  Use  performance  metrics  to  improve  the  perfonnance  of  its  organic 
provider  organizations.  Cross-site  and  Air  Force/extemal,  benchmarking  studies  can 
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promote  sharing  of  best  practices  and  set  performance  goals.  Tracking  perfonnance 
and  holdin-  personnel  accountable  for  meeting  goals  will  be  necessary.  Investment  in 
a  commercial  management  infonnation  system  or  outsourcing  data  collection  to  gen¬ 
erate  desired  infonnation  on  service  outputs  and  processes  may  be  beneficial. 

Innovation  and  Technological  Leadership:  Fifty  Years  of  Competition  in  U.S. 
Aircraft  R&D  (  Rand  RB-52.  1999).  This  research  brief  summarizes  research  more 
fully  documented  in  The  Cutting  Edge:  A  Half  Century  of  U.S.  Fighter  Aircraft  R&D. 
Drawing  on  an  extensive  historical  database  spanning  the  past  50  years  of  U.S.  mili¬ 
tary  aircraft  production  the  authors  identified  three  factors  crucial  to  the  success  of 
fighter  aircraft  R&D:  experience,  competition  and  govermnent-supported  research. 
Downsizing  in  the  aerospace  industry  jeopardizes  these  factors.  Commercial  transport 
development  by  itself  has  not  historically  translated  into  successful  fighter  R&D. 
While  the  commercial  marketplace  may  be  able  to  develop  dual  use  technologies,  it  is 
unlikely  to  produce  methodologies  and  technologies  for  radical  new  developments  in 
military  capabilities.  It  recommends  defense  planners  consider  various  strategies  for 
maintaining  experience  and  promoting  continued  competition  in  the  U.S.  aerospace 
industry: 


Competitive  prototype  and  technology-demonstration  programs 

•  Further  acquisition  refonn 

•  Selective  exploitation  of  the  commercial  industrial  base 

•  Other  innovative  approaches 

The  Cutting  Edge:  A  Half  Century  of  U.S.  Fighter  Aircraft  R&D  (Mark  A. 
Lorell  and  Hugh  P.  Levaux,  Rand  MR-939-AF.  1998).  This  book  reaches  three  gen¬ 
eral  conclusions: 

•  Experience  matters,  because  of  the  tendency  to  specialize  and  thus  to 
develop  system-specific  expertise. 

•  The  most  dramatic  innovations  and  breakthroughs  come  from  secon¬ 
dary  or  marginal  players  trying  to  compete  with  the  industry  leaders. 

•  Dedicated  military  research  and  development  conducted  or  directly 
funded  by  the  United  States  government  has  been  critical  in  the  development  of  new 
higher-performance  fighter  and  bombers 


The  book  recommends  defense  planners  consider  various  strategies  for  main¬ 
taining  experience  and  promoting  continued  competition  in  the  U.S.  aerospace  indus¬ 
try: 

•  Competitive  prototype  and  technology-demonstration  programs 

•  Further  acquisition  refonn 

•  Selective  exploitation  of  the  commercial  industrial  base 

•  Other  innovative  approaches 
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